Joy, pride and emotion

Yesterday, my boss has been awarded the Nobel Prize in economics. The feelings it generates are hard to describe, but over the whole day yesterday I went through a succession of joy, pride and emotion.

Joy when I learned about the award. I was attending Alexandra Roulet's seminar, and right in the middle of the seminar, I heard PhD students shouting and laughing, hi-fiving each other. I turned, ready to ask them to keep a respectful attitude during the talk, but they interrupted me saying "you realy wanna know about this. He won! He won!" "Who won what?" I answered. Despite having formulated the prediction that Jean would win with .15 probability just in the morning, I was so focused on the seminar that I had completely forgotten about the timing of the announcement. "Jean! He won the Nobel prize" they shouted in reply. I was flabbergasted, then overflowed with an intense joy. I apologized to Alexandra, interrupted the seminar and we had a round of applause. Then, we resumed the seminar, but a smile was on everyone's face, and it won't leave us for the rest of the day. This is, I think, what Jean would have wanted. When he was asked about what the prize was going to change in his life, he answered "Nothing, I hope, I like my life as it is, I love research and interacting with colleagues. I wake up every morning happy to go to work. I want this to go on."

When we got out of the room, we could feel that something was different. Everyone was smiling and congratulating each other, despite the fact that we had won nothing for ourselves. Then the word got out that Jean was going to have a press conference. And that's when things started to get crazy. All the media were there, TV stations with their cars equipped with immense antennas, radio, papers. They were interviewing researchers, students, everyone. I might have mumbled stupid words to some mics. The students wore Tshirts with "I love Jean", and they sang "He won the prize, He won the prize". We were all so proud. Later in the evening, when I picked up my kid late at daycare, I apologized, explaining that my boss had just won the Nobel. The nannies looked at me, they smiled and cheered, because they were proud that someone in Toulouse had won this distinction. It is actually the second Nobel prize in Toulouse, after Paul Sabatier more than a hundred years ago.

When Jean entered the press conference room, he smiled, but you could see he was moved. The first thing he did was to hug Colette Laffont, the widow of Jean-Jacques Laffont, the longtime mentor, coauthor and friend of Jean and the founding father of TSE. He was moved to tears. Actually, yesterday, when he was honored by the university for his achievement, he ended his speech by saying that he had a thought for all his colleagues that contributed to the research that the prize celebrates, and "especially for Jean-Jacques Laffont, who, had he not died much too young, would be here, by my side, and would share this prize with me." He almost could not finished his sentence, and he started to cry. It was a deeply moving moment. I was moved to tears, and not the only one in the room. You see, when you work at TSE, even if you have never met him, you can feel the long shadow that Jean-Jacques Laffont casts on this place, almost everyone can share a positive story about Jean-Jacques, and we know what we owe him, that without him, this place would never have existed, this place that, when you get up to go to work every morning, you are happy, and you have a smile on your face, all the more so since last monday. Every morning, I cycle past the Jean-Jacques Laffont roundabout on my way to work, and I was proud before, and I am moved now. This prize crowns a beautiful human adventure, and if you watch closely Jean lecture in Stockholm, I'm sure you'll be able to see the long shadow of Jean-Jacques standing just beside him, smiling and celebrating, as he did the very day when Jean decided to stay definitely in Toulouse with him.

The (Crazy?) Dream of George Udny Yule: Is Non Experimental Causal Inference Feasible?

In 1899, George Udny Yule published a fundamental paper for economics and statistics. In this paper, he proposed to use the method of ordinary least squares to estimate the causal effect of policy variables on outcomes. Actually, the theory was outlined in a 1897 paper and the 1899 paper is the application of the theory of non-experimental causal inference to the estimation of the effect of anti-poverty policies on poverty. By so doing, Yule was setting up a method and a problem that would haunt applied economists for decades and that one hundred years later, is still unsolved.  Some of my own research is trying to revisit this problem. I like Yule's problem because it is an engineering problem and some of the most beautiful engineering results in economics stem from trying to solve this problem, or checking whether the solutions we have do work or not. I also like it because Yule's writing is crystal clear and one would be hard-pressed to try to come up with a cleanest formulation of both the issue and the solution. This is the first post of a series examining the history of non experimental causal inference that will end up with descriptions of some of my current work.

Before detailing the 1899 paper, I would like to quote the introduction to the 1897 paper that is extremely clear on what Yule is doing:

For Yule, the main problem of economists and social scientists is the inability to make experiments. This state of fact has long plagued economics and has lead to the emergence of econometrics. Starting in the 70s, and sometimes even before, experimental methods have started to be used in economics and are now developing at a very large speed.

If you cannot perform experiments, you have to rely on the data of daily experience, a.k.a. observational data. The most pressing problem with observational data is that of confounding factors: factors that simultaneoulsy affect the variable of interest and the outcome variable. Let's be more concrete with Yule's 1899 paper. In this paper, Yule examines whether the strictness of the application of antipoverty laws has an impact on poverty rates. In england at the time were two types of relief offered to the poor: In-relief and Out-relief (See Stephen Stigler's wonderful book for more detail). In-relief or "indoors relief" was given to able bodied individuals taken into workhouses where they would perform various tasks in returns for their subsistence. Out-relief or "out of doors relief" was assistance given in their own homes to persons judged unable to work, e.g. the old or the infirm. A larger proportion of Out-relief was considered to signal a lenient administration.It was clamied at Yule's time that the strictness of the administration was not linked with the poverty rate. In previous work, Yule had shown that this was false and that there was a positive correlation between poverty and the proportion of Out-relief. This seemed to favor a stricter application of the antipoverty laws.

But as Yule was well aware of, correlation is not causation. In his 1899 paper, Yule tackles the difficult task of trying to infer the causal effect of the change in the proportion of Out-relief on the change in poverty. Yule first lists 5 distinct groups of causes that might affect the changes in the rate of poverty:

He then goes on to explain his interest in the first group in a way that no modern economist would deny:

Finally, Yule describes extremely accurately the problem of confounding factors:

An increase in the proportion of the aged, in Yule's example, would simultaneously increase the proportion of Out-relief and the poverty rate. At the same time, the correlation between Out-relief and poverty would be positive but not causal: this is not because there is more Out-relief that there is more poverty, but because there are more elderly. Distinguishing between mere correlation and true causality is a very important task in this case, since we would like to know whether a policy and the way it is implemented can have an effect on poverty, and not whether being older increases poverty. Yule claims that he has an adequate method to solve the problem of confounding factors.

What is this method? Suppose that you could have data on all these variables, says Yule. If you were to use this data to estimate the parameter B in the following simple model, you would have a problem:

 

So what can we do?

Yule argues that by estimating the parameters of this simple multiple linear regression, he can measure the effect of the policy net of all the other influences, ceteris paribus. The regression and econometric adjustment have taken the place of the experiment. By using data and econometrics to control for the influence of the confounding factors, Yule claims that we can conduct causal inference almost as good as in an experiment. Hence, we call this type of approach non-experimental causal inference. The key question obviously is whether the chance of error is now much smaller than before, as Yule argues, or if some missing external influence still messes with the relationship of interest. This is what we will examine in the next posts. The methodological impact of Yule's work has been far-reaching. Today, most social scientists still use the linear model to estimate causal effects. Yule was well aware that the true relationships were not linear, but he argued that he could capture an average effect.

In practice, Yule estimates the parameters of equation (2) above by minimizing the sum of the squared residuals of this equation. He finds the following result:

An increase in Out-relief of 10 percent increases poverty by roughly 3 percent. Thus the positive raw correlation between Out-relief and poverty did not seem to mask a negative relationship. The size of the effect is much smaller though after accounting for the porportion of old and the population. Indeed, the raw correlation coefficient was .38, almost a third more. Thus, multiple regression enables one to pin down more efficently and precisely causal effects.

Yule's paper is extremely rich and any student could take inspiration from it. For example, Yule uses his estimates to decompose the total variation in poverty rate between factors due the policy and other factors:

We can wee that even if unkwown factors were responsible for the majority of the poverty decrease in rural and mixed areas, the increase in the severity of the relief policy (and the resulting decrease in Out-relief) have been a major factor in decreasing the urban poverty rate, at least between 1871 and 1881.

Yule's beautiful analysis withstand the test of time. The main question for its validity is whether the important factors that have been left out and that account for a majority of the variation in poverty are also correlated with Out-relief. If this is the case, they are still biasing Yule's estimates. This is the key question that nonexperimental causal inference has to answer. That is, we still do not know whether the method that Yule has devised is really competent to extract causal relationships from the data of daily experience.

I've also just become aware that Yule can be credited as being the inventor the the Difference In Difference Matching method that I study in my own work. More on this in subsequent posts.

History of econometrics: special conference session at the EEA-ESEM meeting

One of my secret manias (and one that I am less ashamed to admit) is an interest for the history of econometrics. With such an inclination, the two successive sessions of the EEA-ESEM meeting on the history of econometric thought were obviously an absolute delight to me. Here, I am going to try to convey some of the excitement that I got out of this sessions by giving a quick summary of the papers presented. This post will also appear in the next issue of the TSE Mag.

In the first talk of the first session, John Aldrich detailed how the father of modern econometrics  Trygve Haavelmo, contributed to the understanding and formalization of the concept of causality. John especially emphasized the modernity of Haavelmo's views on this topic and how they have resurfaced in more recent literature. Marcel Boumans described the interaction between Milton Friedman and the members of the Cowles Commission, the founders of modern econometrics. Both interacted in the late forties in Chicago. Marcel gave a fascinating account of how much Friedman was involved into methodological debates at the time, that eventually yielded to the writing of his essays in positive economics. Friedman was at the time extremely critical of the strong theorisation of econometrics promoted but the then Commission president Tjalling Koopmans (Marcel quoted from an unpublished assessment by Friedman of the works of the Cowles commission). At some point, Friedman defied Koopmans and one of his boys, Lawrence Klein, to check whether the complex macro simultaneous equations model estimated on pre-war data could predict macro conditions in 1948. This was a dismal failure and yielded Klein to reassess his model. Marcel put this debate in perspective by recalling how it related to the views of early economists Alfred Marshall on one side and Léon Walras on the other. Till Dueppe presented a thorough account of a trip taken by Tjalling Koopmans to the Soviet Union in 1976. Koopmans was really eager to discover if the linear programming methods developed by Leonid Kantorovich were applied in practice by the planning bureau of the Soviet Union. What he found was a very dispirited Kantorovitch, saying that they might have read about it, at least it was somewhere in published form. Koopmans was also somewhat disappointed by the quality of economics research that he found there. He was on the other hand extremely impressed to meet with some of the most outstanding mathematicians of the world.

The next session started with a minutely accurate detail of the first meeting of the econometrics society in 1931 in Lausanne by Olav Bjerkholt. What striked me the most was the impressive imprint that Ragnar Frish left on this meeting and on the early years of the econometric society. While François Divisia, the Vice President officially in charge of organizing the meeting was preoccupied by such urgent matters as choosing the suitable French translation of "econometrician," Frish used his spare time while recovering from a skiing accident to write a sketch of a programme and one of the three papers he presented at the conference. Frish also insisted that the conference opened with a presentation of some of the works of six of the founding fathers of economics (Marshall was not one of them). This echoes evidence given by Marcel Boumans that early Econometrica issues published handwritten notes by Walras, along with his correspondence with other economists. Duo Qin advocated the return of the use of the terms autonomy and confluence in modern econometrics textbooks. These terms have been coined by Frish to separate what we now call structural and reduced form relationships. These late notions were in fact derived from the earlier notions by Tjallling Koopmans. The great advantage of the term "autonomy" is that it puts emphasis on the fact that we are looking for invariant relationships that remain true when the other relations in the economy change. The Lucas critique is in a sense a mere application of the notion of autonomy, which Lucas acknowledged explicitly in his paper (see the reference to Marschak in footnote 3). Cléo Chassonery-Zaïgouche presented the evolution over the years of the empirical analysis of discrimination. She especially showed that the traditional econometric techniques developed in the 70s in the Cowles tradition are progressively replaced by experiments. This trend is not apparent in the number of papers published in top journals though, but is clear in the influence of the published papers: experiments gain much more citations. Finally Jan Höffler presented an exciting collaborative wiki project on replication in economics. With his colleagues, he has set up a list of the published papers in top journals with a link to the data sets and code. With his students, they are now trying to replicate the authors' findings. Being a collaborative project, anyone can report the results of his own replication exercise on the web page. An exciting exercise for students in econometrics.

Let's open the scientific kitchen!

I am an academic economist and I am starting a blog. What I would like to do is to take the prospective readers (if any) throughout the journey of a young academic economist trying to advance economic science along with his own career.

For me, this blog would serve as a sort of lab notebook, where I register my ideas, thoughts, trials and errors and eventual results. This would be very helpful for me to keep track of my thought process, how I came to my results. It will also serve to polish my understanding and explaining of what I'm doing along the way, something that I think will prove critical when writing the final paper. I also would like this blog to be a sort of disciplinary device incentivizing me not to procrastinate too much on a project, by making explicit how much time has elapsed since I said I would do something. On the downside, one could say that blogging is the ultimate procrastination device. I hope not, and I want to see whether it works. Finally, I will indulge occasionally in one of my sinful pleasures: methology and history of science, especially of economics.

One of my hopes is that this blog gives a picture of the inner workings of my work, and thus contributes to give the audience a better understanding of how scientists in general, and economists in particular, work. Terence Tao's blog about his mathematical research is a beautiful example of what one can achieve in this respect. My hope is that in this way, we can overcome the occasional distrust that citizens have towards scientific work by opening up the machinery of what we are doing. This is a similar undertaking as the open kitchen movement in restaurants. Clients like it because it signals that the chef has nothing to hide, he does everything in the open, he has nothing to be ashamed of. Blogging opens the box of one's own methods and choices and hopefully participates to convince nonscientists that we have nothing to hide, and that everything is in the open for them to see. Maybe in the future, with more scientific blogging, we can decrease the likelihood of controversies such as the Climatic Research Unit email controversy or the one about the reproducibility of psychological research.

In order to start this blog, I have to overcome my own "academic paranoia", the fear that my unfinished ideas get stolen. Time and again, I've had this feeling that I am not particularly proud of, the reluctance to expose budding ideas publicly, for the fear that someone might get inspired by them and write a paper in the meantime. There is also the related fear of triggering an attrition war. If someone hears about me working on some topic, he might hurry to finish a related work. Think of Andrew Wiles locking himself up in his attic to crack Fermat's last theorem, after accumulating a bundle of papers to send each year for publication, in order to distract possible competitors and convince them that he was not working on proving Fermat's last theorem but still following his old research path, as Simon Singh reports. Related is the fear of a precedence war to decide who has come with an idea first. Michel Moreaux told me that Econometrica received 2 papers proving general equilibrium theory within 6 months in 1960 1952. There is actually a recent book on this topic. A few weeks ago a colleague told me that he received an email of a researcher that has conducted a very similar research, and that he did not know how to react.

 

My feeling is that being more open about which research we are pursuing since the start might help to make it clearer who contributed to what on a given topic. One very nice way to establish priority is to have blog entries showing the way your research unfolded. Tim Gowers made some excellent points on this topic when he launched the collaborative Polymath project. My experience is that sharing ideas early on generally makes me clarify them and has a major effect on the direction I eventually take. It might also help to identify potential collaborations very early along the way, etc.

Let's tame the fear then and let me take you for a journey in my scientific kitchen.