TestMagic

fp3690 · 02-11-2008, 07:02 PM

I just stumbled upon this Generalized Maximum Entropy Home Page.

Anyone knows what maximum entropy is?

macroeconomicus · 02-11-2008, 07:10 PM

No idea, something from Physics.. there is also a related book

Amazon.com: Maximum Entropy Econometrics: Robust Estimation with Limited Data: Books: Amos Golan,George G. Judge,Douglas Miller

filroz · 02-11-2008, 08:00 PM

It is connected with second law of thermodynamics. It can be interpreted as a measure of chaos (or order) in the system. More order, lower entropy and vice versa.
For example you have two bowls, in one there are only red balls, white in the second. This system is very "organised". If you mix it, and randomly put the balls back into bowls, the colors will be mixed and the system will have much lower degree of "order" or higher entropy.
This "cause" that some natural processes are irreversible. For example, if oyu have two objects with different temperature, then you can increase the temperature of the colder but not the warmer, even though the total energy would be the same.
If you let your room, then there will be an inevitable tendency towards mess, which can be avoided only by some work

Andronicus · 02-11-2008, 10:17 PM

From our future artificial overlord:

Principle of maximum entropy - Wikipedia, the free encyclopedia

polkaparty · 02-11-2008, 10:40 PM

Quote:

Originally Posted by Andronicus

From our future artificial overlord

I have really come full circle on wikipedia. It's great for quick information but it is replete with errors and misinformation as well. Worse, if you're reading the article, you probably don't know enough to notice the errors. I have found the combination of amazon.com (to discover new books) and my university library to be a lethal combination. (lethal to the problem at hand that is....)

Furthermore, few people seem to use the library extensively these days. I have a collection of about 50 library books which I have checked out over the years. Some I have had for 3 years now. [my library allows perpetual renewal as long as there isn't an outstanding request]. Unfortunately when I graduate all my precious books will have to good back to their cold former lives on the stacks....

Julius · 02-11-2008, 11:56 PM

I happen to know some about this cuz I've been working on this for a while. Followings are some random bits of my knowledge. May contain a lot of errors.

1. 'entropy' here means degree of unorderedness(or chaos), as in thermodynamics.

2. the concept of so called maximum entropy principle, however, has been developed since 1940's by a group of people studying statistical dynamics. Shannon defined the measure of entropy for a probability density, which of course can be derived axiomatically with 'good' properties.

3. Maximum entropy principle provides a way to 'estimate' (I would rather say 'characterize') probability density with only limited information on hand. That is, to find a pdf that maximizes entropy under the constraint of information on hand. The information here is typically given as a form of moment conditions.

4. As you may notice, the maximum entropy principle can be applied to the same problems that GMM can approach. Kitamura and Stutzer(guess it's 1997 econometrica) proposed a semiparametric estimation method using maximum entropy principle. Their method is basically the maximum empirical likelihood estimation, but with empirical likelihood probabilities obtained by maxmimum entropy principle.

5. There are many other application of the maxent principle. It seems like it is mostly applied in finance literature, extending ARCH type models, obtaining state price density and so on.

roy1122 · 03-13-2008, 05:31 PM

hi julius,

thanks for sharing the concepts of GME.
if i have a ready pdf (obtained by plotting exptl data set) of the frequency of occuerence of a particular event (in a time series), how can i calculate the GME of my pdf ?
can u give some more details about it. also, some easy papers/books (preferably easily found in internet) would be of great help t start for me. i am not from a u pure economics background.

thanks & hoping for ur reply.

PBR · 03-14-2008, 04:42 AM

This is a total aside, but I can't help it: as I was reading Critical Mass, I stumbled on these considerations on enthropy and the like. I readily thought that sooner or later some guy would translate this into an economic concept, just like we did with Mechanics. It was already done as I can see now.

roy1122 · 03-20-2008, 02:34 PM

hi,

can anyone inform me how can we apply GME to our problem ? what are the basic equations and the methodology to be followed ?

thanks in advance.

02-11-2008, 07:02 PM	#1 (permalink)
fp3690 TestMagic Guru-in-Training Join Date: Feb 2007 Posts: 521	Maximum Entropy I just stumbled upon this Generalized Maximum Entropy Home Page. Anyone knows what maximum entropy is?

Thread Tools	Search this Thread
Show Printable Version Email this Page	Search this Thread: Advanced Search
Display Modes
Linear Mode Switch to Hybrid Mode Switch to Threaded Mode

02-11-2008, 07:10 PM	#2 (permalink)
macroeconomicus Within my grasp! Join Date: Sep 2007 Posts: 329	No idea, something from Physics.. there is also a related book Amazon.com: Maximum Entropy Econometrics: Robust Estimation with Limited Data: Books: Amos Golan,George G. Judge,Douglas Miller

02-11-2008, 08:00 PM	#3 (permalink)
filroz Driving on the left... Join Date: Jan 2007 Location: London, UK Posts: 366	It is connected with second law of thermodynamics. It can be interpreted as a measure of chaos (or order) in the system. More order, lower entropy and vice versa. For example you have two bowls, in one there are only red balls, white in the second. This system is very "organised". If you mix it, and randomly put the balls back into bowls, the colors will be mixed and the system will have much lower degree of "order" or higher entropy. This "cause" that some natural processes are irreversible. For example, if oyu have two objects with different temperature, then you can increase the temperature of the colder but not the warmer, even though the total energy would be the same. If you let your room, then there will be an inevitable tendency towards mess, which can be avoided only by some work

02-11-2008, 10:17 PM	#4 (permalink)
Andronicus The better metal snake! Join Date: Aug 2007 Posts: 353	From our future artificial overlord: Principle of maximum entropy - Wikipedia, the free encyclopedia

02-11-2008, 11:56 PM	#6 (permalink)
Julius 2008 Econ Ph.D. Join Date: Aug 2007 Posts: 24	I happen to know some about this cuz I've been working on this for a while. Followings are some random bits of my knowledge. May contain a lot of errors. 1. 'entropy' here means degree of unorderedness(or chaos), as in thermodynamics. 2. the concept of so called maximum entropy principle, however, has been developed since 1940's by a group of people studying statistical dynamics. Shannon defined the measure of entropy for a probability density, which of course can be derived axiomatically with 'good' properties. 3. Maximum entropy principle provides a way to 'estimate' (I would rather say 'characterize') probability density with only limited information on hand. That is, to find a pdf that maximizes entropy under the constraint of information on hand. The information here is typically given as a form of moment conditions. 4. As you may notice, the maximum entropy principle can be applied to the same problems that GMM can approach. Kitamura and Stutzer(guess it's 1997 econometrica) proposed a semiparametric estimation method using maximum entropy principle. Their method is basically the maximum empirical likelihood estimation, but with empirical likelihood probabilities obtained by maxmimum entropy principle. 5. There are many other application of the maxent principle. It seems like it is mostly applied in finance literature, extending ARCH type models, obtaining state price density and so on.

03-13-2008, 05:31 PM	#7 (permalink)
roy1122 Trying to make mom and pop proud Join Date: Mar 2008 Posts: 2	hi julius, thanks for sharing the concepts of GME. if i have a ready pdf (obtained by plotting exptl data set) of the frequency of occuerence of a particular event (in a time series), how can i calculate the GME of my pdf ? can u give some more details about it. also, some easy papers/books (preferably easily found in internet) would be of great help t start for me. i am not from a u pure economics background. thanks & hoping for ur reply.

03-14-2008, 04:42 AM	#8 (permalink)
PBR --- Join Date: Feb 2008 Posts: 37	This is a total aside, but I can't help it: as I was reading Critical Mass, I stumbled on these considerations on enthropy and the like. I readily thought that sooner or later some guy would translate this into an economic concept, just like we did with Mechanics. It was already done as I can see now.

03-20-2008, 02:34 PM	#9 (permalink)
roy1122 Trying to make mom and pop proud Join Date: Mar 2008 Posts: 2	hi, can anyone inform me how can we apply GME to our problem ? what are the basic equations and the methodology to be followed ? thanks in advance.

TestMagic

Free GMAT, GRE, SAT Test, TOEFL practice tests