Char­ac­ter­is­tic Func­tions and Sta­ble Dis­tri­b­u­tions

In the pre­vi­ous post, I ex­plored the use of the gen­er­al­ized nor­mal dis­tri­b­u­tion to mod­el the price move­ments of fi­nan­cial in­stru­ments. This ap­proach of­fered bet­ter fit­ting dis­tri­b­u­tions than the nor­mal and Laplace dis­tri­b­u­tions stud­ied in ear­li­er post­s. But the shape of the fit­ted dis­tri­b­u­tions still did­n’t quite match the shape of the his­togram. In this post, I want to ex­plore a class of prob­a­bil­i­ty dis­tri­b­u­tions known as Lévy al­pha-stable dis­tri­b­u­tion­s. And to ex­plore these dis­tri­b­u­tion­s, we need to un­der­stand char­ac­ter­is­tic func­tion­s.

Char­ac­ter­is­tic Func­tions

In my mind, the most in­tu­itive rep­re­sen­ta­tion of a prob­a­bil­i­ty dis­tri­b­u­tion is a prob­a­bil­i­ty den­si­ty func­tion. But there are oth­er pos­si­ble ways to rep­re­sent a prob­a­bil­i­ty dis­tri­b­u­tion. You could, for ex­am­ple, use a cu­mu­la­tive dis­tri­b­u­tion func­tion in­stead. An­oth­er al­ter­na­tive is to use a char­ac­ter­is­tic func­tion. The char­ac­ter­is­tic func­tion of a prob­a­bil­i­ty dis­tri­b­u­tion is just the Fouri­er trans­form of its den­si­ty func­tion:

As shown above, if you know the for­mu­la for the prob­a­bil­i­ty den­si­ty func­tion, you can com­pute the char­ac­ter­is­tic func­tion by eval­u­at­ing the in­te­gral. Like­wise, if you know the for­mu­la for the char­ac­ter­is­tic func­tion, you can work out the prob­a­bil­i­ty den­si­ty func­tion by tak­ing the in­verse trans­for­m, as shown be­low:

There are a va­ri­ety of dif­fer­ent pur­pos­es for which char­ac­ter­is­tic func­tions can be use­ful. Char­ac­ter­is­tic func­tions can be used to sim­pli­fy the con­vo­lu­tion op­er­a­tion, for ex­am­ple. There are al­so prob­a­bil­i­ty dis­tri­b­u­tions that can be clean­ly rep­re­sent­ed us­ing char­ac­ter­is­tic func­tions but do not have prob­a­bil­i­ty den­si­ty func­tions that can be ex­pressed in closed for­m.

Sta­ble Dis­tri­b­u­tions

Ex­cept for a small num­ber of cas­es, the Lévy al­pha-sta­ble dis­tri­b­u­tion does not have a prob­a­bil­i­ty den­si­ty func­tion that can be ex­pressed an­a­lyt­i­cal­ly. How­ev­er, it is pos­si­ble to rep­re­sent the prob­a­bil­i­ty den­si­ty func­tion in terms of the char­ac­ter­is­tic func­tion. Here is the gen­er­al form of the char­ac­ter­is­tic func­tion for stable dis­tri­b­u­tion­s:

The lo­ca­tion pa­ra­me­ter can be neg­a­tive or pos­i­tive. The scale pa­ra­me­ter is al­ways a pos­i­tive num­ber. The shape pa­ra­me­ter must be a pos­i­tive num­ber no greater than two, while the skew pa­ra­me­ter can range from neg­a­tive one to pos­i­tive one. For the sake of sim­plic­i­ty, let’s drop the skew pa­ra­me­ter and as­sume the skew is al­ways ze­ro. Omit­ting the skew pa­ra­me­ter sim­pli­fies the char­ac­ter­is­tic func­tion to the fol­low­ing:

This is the char­ac­ter­is­tic func­tion for the sym­met­ric al­pha-stable dis­tri­b­u­tion. The cor­re­spond­ing prob­a­bil­i­ty den­si­ty func­tion is sym­met­ric with re­spect to the lo­ca­tion pa­ra­me­ter. Us­ing the char­ac­ter­is­tic func­tion above, we can com­pute the prob­a­bil­i­ty den­si­ty func­tion like this:

We can use nu­mer­i­cal in­te­gra­tion to eval­u­ate this func­tion and plot charts for spe­cif­ic pa­ra­me­ter val­ues. Hold­ing the lo­ca­tion and scale pa­ra­me­ters con­stant and vary­ing the shape pa­ra­me­ter, we can see what the shape of the den­si­ty func­tion looks like for dif­fer­ent val­ues of the shape pa­ra­me­ter. Here are some il­lus­tra­tions:

There are two spe­cial cas­es in the ex­am­ples il­lus­trat­ed above. When the shape pa­ra­me­ter is equal to two, the sta­ble dis­tri­b­u­tion is equiv­a­lent to a nor­mal dis­tri­b­u­tion. When the shape pa­ra­me­ter is equal to one, the sta­ble dis­tri­b­u­tion is equiv­a­lent to a Cauchy dis­tri­b­u­tion. For these two spe­cial cas­es, the prob­a­bil­i­ty den­si­ty func­tion can be ex­pressed in closed for­m.

Nor­mal Dis­tri­b­u­tion

The nor­mal dis­tri­b­u­tion is a spe­cial case of a sta­ble dis­tri­b­u­tion when the shape pa­ra­me­ter equals two. Plug the fol­low­ing val­ues in­to the char­ac­ter­is­tic func­tion:

With these val­ues, the char­ac­ter­is­tic func­tion now looks like this:

And we can now find the prob­a­bil­i­ty den­si­ty func­tion based on the char­ac­ter­is­tic func­tion:

Eval­u­at­ing the in­te­gral, we can ar­rive at the fol­low­ing ex­pres­sion of the nor­mal dis­tri­b­u­tion:

As shown above, by hold­ing the shape pa­ra­me­ter to a fixed val­ue of two, the den­si­ty func­tion of the sta­ble dis­tri­b­u­tion is equiv­a­lent to that of a nor­mal dis­tri­b­u­tion.

Cauchy Dis­tri­b­u­tion

The Cauchy dis­tri­b­u­tion is a spe­cial case of a sta­ble dis­tri­b­u­tion when the shape pa­ra­me­ter equals one. Plug the fol­low­ing val­ues in­to the char­ac­ter­is­tic func­tion:

With these val­ues, the char­ac­ter­is­tic func­tion now looks like this:

And we can now find the prob­a­bil­i­ty den­si­ty func­tion based on the char­ac­ter­is­tic func­tion:

Eval­u­at­ing the in­te­gral, we can ar­rive at the fol­low­ing ex­pres­sion of the Cauchy dis­tri­b­u­tion:

As shown above, by hold­ing the shape pa­ra­me­ter to a fixed val­ue of one, the den­si­ty func­tion of the sta­ble dis­tri­b­u­tion is equiv­a­lent to that of a Cauchy dis­tri­b­u­tion.

Nu­mer­i­cal Pa­ra­me­ter Es­ti­ma­tion

We can use the max­i­mum like­li­hood method in the same fash­ion we did in the pre­vi­ous post to fit the lo­ca­tion, scale, and shape pa­ra­me­ters to a sta­ble dis­tri­b­u­tion. Here is the like­li­hood func­tion that we want to max­i­mize:

To fit a sta­ble dis­tri­b­u­tion to an ob­served set of data, we need to find the pa­ra­me­ter val­ues that max­i­mize this func­tion. For the ex­am­ples ap­pear­ing lat­er in this post, we’re go­ing to use nu­mer­i­cal op­ti­miza­tion to find the most op­ti­mal pa­ra­me­ter val­ues. The cost func­tion we’re go­ing to use is the nega­tion of the log­a­rithm of the like­li­hood func­tion:

Us­ing nu­mer­i­cal op­ti­miza­tion meth­od­s, we need to start with an ini­tial guess. As with the pre­vi­ous post, I’ve cho­sen to start with a shape pa­ra­me­ter val­ue of two:

This im­plies a nor­mal dis­tri­b­u­tion. The lo­ca­tion and scale pa­ra­me­ters for a nor­mal dis­tri­b­u­tion are easy to com­pute:

These are the ini­tial pa­ra­me­ter val­ues for the nu­mer­i­cal op­ti­miza­tion method. Like be­fore, I’m us­ing a third-par­ty im­ple­men­ta­tion of the Nelder–Mead al­go­rith­m. This is an un­con­strained method, so it’s pos­si­ble for the op­ti­miza­tion process to stray out­side the range of valid pa­ra­me­ter val­ues. To off­set this, I’ve mod­i­fied the cost func­tion to re­turn a penal­ty val­ue if the op­ti­miza­tion pro­ce­dure wan­ders out­side the valid range of val­ues for the shape pa­ra­me­ter.

Nu­mer­i­cal In­te­gra­tion

In ad­di­tion to us­ing nu­mer­i­cal op­ti­miza­tion to find the op­ti­mum pa­ra­me­ter val­ues, we al­so need to use nu­mer­i­cal in­te­gra­tion to eval­u­ate the prob­a­bil­i­ty den­si­ty func­tion for sta­ble dis­tri­b­u­tion­s. I’m us­ing an im­ple­men­ta­tion of the Gauss–Kro­n­rod method for the ex­am­ples in this post. Since nu­mer­i­cal in­te­gra­tion is a com­pute-in­ten­sive eval­u­a­tion method, it can be ben­e­fi­cial to con­sid­er the run­time per­for­mance of the eval­u­a­tion of the in­te­grand. Con­sid­er the fol­low­ing:

The in­te­grand has a com­plex ex­po­nen­tial, so we can use Euler’s for­mu­la to sep­a­rate the re­al part from the imag­i­nary part. And if we can as­sume the fi­nal re­sult is al­ways a re­al num­ber, we can sim­ply drop the imag­i­nary part en­tire­ly:

No­tice that, in the for­mu­la above, the in­te­grand is sym­met­ric with re­spect to ze­ro. That means we can do a lit­tle trick where we just in­te­grate over the pos­i­tive half of the in­ter­val and then dou­ble the re­sult:

With the den­si­ty func­tion for the sta­ble dis­tri­b­u­tion in this for­m, the run­time per­for­mance seems to be im­proved by a fac­tor of five. There are prob­a­bly oth­er op­ti­miza­tions that can be made as well, but this is fast enough for the ex­am­ples in this post.

Mi­crosoft Stock Prices

Now let’s look at some ex­am­ples of fit­ting sta­ble dis­tri­b­u­tions to re­al-life da­ta. The first ex­am­ple is based on the his­tor­i­cal stock prices of Mi­crosoft Cor­po­ra­tion. Like we did in the last post, we’ll take the log­a­rithm of the dai­ly clos­ing prices, com­pute the first dif­fer­ences, and then put the data in a his­togram. The fol­low­ing charts show the his­togram over­laid with the fit­ted den­si­ty func­tions for the nor­mal, Cauchy, and gen­er­al­ized sym­met­ric al­pha-stable dis­tri­b­u­tion­s, re­spec­tive­ly:

The fit­ted sym­met­ric al­pha-stable dis­tri­b­u­tion has the fol­low­ing shape pa­ra­me­ter:

As ex­pect­ed, the den­si­ty func­tion for the gen­er­al­ized form of the sta­ble dis­tri­b­u­tion ap­pears to be a bet­ter fit than that of ei­ther the nor­mal dis­tri­b­u­tion or the Cauchy dis­tri­b­u­tion. Worth not­ing, how­ev­er, is that the fit­ted curve does­n’t seem to fit the his­togram as tight­ly as the gen­er­al­ized nor­mal den­si­ty func­tion stud­ied in the pre­vi­ous post—sub­jec­tive­ly speak­ing, any­ways.

Bit­coin Prices

This ex­am­ple us­es his­tor­i­cal bit­coin prices. We’ll do the same thing we did in the pre­vi­ous ex­am­ple, tak­ing the first dif­fer­ences of the log­a­rithm of the dai­ly clos­ing prices and putting the da­ta in a his­togram. The fol­low­ing charts show the his­togram over­laid with the fit­ted den­si­ty func­tions for the nor­mal, Cauchy, and gen­er­al­ized sym­met­ric al­pha-stable dis­tri­b­u­tion­s, re­spec­tive­ly:

The fit­ted sym­met­ric al­pha-stable dis­tri­b­u­tion has the fol­low­ing shape pa­ra­me­ter:

In this ex­am­ple, the fit­ted den­si­ty func­tion for the Cauchy dis­tri­b­u­tion looks like a bet­ter fit than the den­si­ty func­tion for the gen­er­al­ized form of the sta­ble dis­tri­b­u­tion. This is not what I was ex­pect­ing. The Cauchy dis­tri­b­u­tion is a spe­cial case of the sym­met­ric sta­ble dis­tri­b­u­tion, so the sta­ble dis­tri­b­u­tion should fit at least as well as the Cauchy dis­tri­b­u­tion. But that does­n’t seem to be the case here. I am both­ered by this re­sult.

Nat­ural Gas Prices

The third ex­am­ple us­es the dai­ly clos­ing prices of a nat­ural gas ET­F. Once again, we’ll take the log­a­rithm of the price quotes, com­pute the first dif­fer­ences, and then put the da­ta in­to a his­togram. The fol­low­ing charts show the his­togram over­laid with the fit­ted den­si­ty func­tions for the nor­mal, Cauchy, and gen­er­al­ized sym­met­ric al­pha-stable dis­tri­b­u­tion­s, re­spec­tive­ly:

The fit­ted sym­met­ric al­pha-stable dis­tri­b­u­tion has the fol­low­ing shape pa­ra­me­ter:

In this case, the den­si­ty func­tion for the gen­er­al­ized sta­ble dis­tri­b­u­tion seems to fit quite nice­ly. It looks like a bet­ter fit than the den­si­ty func­tion for nor­mal dis­tri­b­u­tion or the Cauchy dis­tri­b­u­tion. And the shape of the den­si­ty func­tion looks like a bet­ter match than that of the gen­er­al­ized nor­mal dis­tri­b­u­tion used in the last post.

Bet­ter Meth­ods

I am not hap­py with the re­sults of this ex­per­i­men­t. Com­pared to the gen­er­al­ized nor­mal dis­tri­b­u­tion used in the last post, the shape of the den­si­ty func­tion for the Lévy al­pha-sta­ble dis­tri­b­u­tion seems to be a bet­ter fit for the kind of da­ta used in the ex­am­ples above. How­ev­er, fit­ting the pa­ra­me­ters us­ing the max­i­mum like­li­hood method does­n’t al­ways seem to give the best re­sult­s. I am par­tic­u­lar­ly dis­ap­point­ed with the re­sults based on the bit­coin price da­ta. And I think there might be bet­ter fit­ting pa­ra­me­ters for the oth­er da­ta sets as well. Since stable dis­tri­b­u­tions have heavy tail­s, I sus­pect that a dearth of sam­ple data deep in the tails might be throw­ing off the pa­ra­me­ter es­ti­mates. It might be worth do­ing a fol­low-up study to ex­per­i­ment with dif­fer­ent fit­ting meth­ods that might yield a bet­ter re­sult.

Ac­com­pa­ny­ing source code is avail­able on GitHub.