GiveWell Panel

Earlier this week, I attended a panel discussion at GiveWell that focused on issues surrounding the Randomized Control Trial (RCT) movement within the non-profit/NGO sector, which is fancy way of saying that organizations should have some evidence that their programs work before trying to scale them up.

Before I share my notes, let me share some context. Like many other fields, there is a greater demand and push for data-driven and evidence-based solutions. Clearly not everything can be measured and there is room for experimentation and unproven ventures, yet there are a lot of organizations that cannot reliably prove that they have much (if any) impact relative to the status quo counterfactual.

An implicit premise of the discussion was that a lot of RCT data is dirty and results from similar trials can be highly variable. This is not surprising, since much of this data is from impoverished locales.

There was some debate over the bias towards studies that illustrate globally consistent conclusions vs  recognizing that geo-specific differences can and do exist. There was also a debate over bottom-up grassroots work vs top-down public policy solutions.

Even in charitable, intervention, and development programs, there are no sure bets. Having a 95% confidence level in what you’re doing is unrealistic (95% falls within two standard deviations and is often accepted as a sure thing). Even 80% confidence levels may be too high a bar. Panelists agreed that there is no hard and fast threshold for supporting a program, but confidence levels, cost, practical considerations, and externalities should be considered.

Below are several other comments that I found interesting:

  • It is difficult, if not impossible, to separate a program’s effectiveness from the organization running it. 
  • Data collection could be a lot better because much of it is driven by academics who have different incentives that decision-makers and practitioners.
  • A case can be made to support a less organized organization with poor data monitoring rather than an organization with great data, if the there’s a wide disparity in the estimated impact (ie. deworming programs can increase incomes more than cash transfer programs by a factor of 10x)
  • Some of the biggest surprises the panelists have seen has been behavioral incentives having outsized positive impact (ie. immunization rates shooting up dramatically if you offer a kilo of lentils)

Overall, I enjoyed the event. Rather than an overconfidence in RCTs and data, the panelists were upfront about the limitations and misuses of RCT data. This surprised me since GiveWell has built their reputation on data-driven giving. To some extent, philanthropy is like anything else: you make decisions about an uncertain future with incomplete and imperfect information.

A Dozen Things I Learned in Cuba

“We don’t go on vacations. We go on your field trips.”  -my wife, pictured above in Habana Vieja

[a repost from FB, based on the positive feedback I received and the fact that many of the below points are economic in nature]

I’m going to channel my inner Tren Griffin and share 12 things that I learned in Cuba, based both on my observations and on the handful of conversations that exceeded 15-30 minutes. A small sample size and subjective, so take it for what it’s worth. However, I tried to ask different people many of the same questions for corroboration and to understand the nuances and variety of perspectives.

1. Almost everyone works for the government and earns $20-40 per month, which is not enough to cover basic needs in Cuba. I start with this fact because it has many implications and helps explain many of the following points.

2. Many people steal from their jobs to survive. If you buy 4 cups of rice, you’ll only get 3 cups. If you buy a bottle of cooking oil, it will only be partially full. Everyone accepts this. The missing amounts are sold by employees to supplement their meager incomes. Other people operate side businesses by reselling the stolen goods or black market goods brought in on commercial flights (like Colgate toothpaste, which is much better than what can be found in stores).

3. The government provides free education and medical care. The general consensus seems to be that it was decent until the USSR (and it’s subsidies) collapsed. The quality of education and medicine is pretty low and one person said that teachers and doctors will provide mediocre service unless paid under the table.

4. Service is generally slow and terrible/non-existent. Since the government runs nearly everything, there is very little competition or profit-incentive, and everyone gets paid the same regardless of skill or effort. Many Cubans lamented how lazy this makes the majority of employees. Service at private businesses is markedly better, to the point that you can pretty easily distinguish a government-run business from a privately-run one based solely on quality of service.

5. There is a serious lack of resources. People complain that they cannot find the basic things that they need. They may spend a few hours per day visiting multiple stores before being able to find basic staples like rice or oil. Most stores I saw just had empty shelves. Even as a tourist eating at the most expensive restaurants, many menu items were often unavailable (a couple of times there were only one or two dishes available, out of a menu of 10-20 items). We had a couple great meals, but the food quality was pretty low overall.

6. American tourists are flooding the island. Flights from the US started just a few months ago, but it’s already bringing an unprecedented number of tourists. People shared that they have never seen the numbers of visitors that they’ve seen in the past few months. There is a consensus that US tourism will be massive, but there also seems to be a lot of fear of what changes it will bring. While tourism is booming, so is food and rent inflation. The business owners I spoke to said business is better than ever and exceeding all expectations. Yet, they also shared that Cubans are getting priced out of housing, goods, and services.

7. People do not keep their money in banks. For one thing, Cubans are worried the government may confiscate or revalue the currency (which is roughly pegged to USD). Technically, it’s only legal to exchange currencies at official exchanges, but many Cubans will offer better rates for hard currencies on the black market so they can accumulate currency that will hold its value better (both the exchanging and owning of foreign currencies is illegal for Cubans). Secondly, bank tellers will ask for the source of any deposits. Since the majority of people’s income is from black market activity, they cannot deposit funds in the bank.

8. Wealth is not flaunted. In fact, it’s deliberately hidden. If a Cuban took multiple day trips or overnight trips around the country (like we did), the government may knock on your door, interrogate you and investigate the source of your money. Even licensed private business owners keep a very low profile. All expressed a fear that success would bring attention and that the government would shut down their business. Many people mentioned that they are starting make more money, but they cannot enjoy it at all for fear of bringing attention to themselves.

9. Despite the progress, people are not as optimistic or confident as I expected. Everyone I spoke to mentioned that although things are improving lately, the government can roll all the reforms back in an instant. They’ve done it before and they may do it again.

10. Despite the party line, there is an economic hierarchy and racial issues. I spoke to someone that said they didn’t like black people. I spoke to a black person who said he’s been discriminated against.

11. A couple people mentioned the old, fat men that come to Cuba for young girls. Both mentioned an hour costs $20, which I took to mean that it’s common enough that the average person knows the standard price. One person pointed out a schoolgirl walking by and said “She’s looking for men.” The girl looked like normal adolescent kid in a school uniform and I asked how they could be sure. “Do you want me to prove it?” my acquaintance replied. I passed.

12. There is clearly a wealth gap between Cubans and foreigners, but nobody tried to guilt us or ask for money. We did pay foreigner premiums and people took commissions for helping us arrange things, but we only ran into one insistent tout/jintero and two people asked me for food. We did see many street performers attempting to earn money. Cubans are proud and not looking for any handouts, IMO.

People were very friendly and happy to chat. I feel fortunate to have visited and met so many people. Although I probably won’t visit again, I had a good time and learned a lot.  Viva Cuba!