This page logs mistakes we've made, strategies we should have planned and executed differently, and lessons we've learned.

Because we are a startup organization working in areas we have little experience with, it is particularly important that we constantly recognize and learn from our shortcomings. We make this log public so as to be up front with any potential supporters about ways in which we need to improve.

A full review of our first year – both accomplishments and shortcomings - is available here.

Please use our survey, or just contact us, with other items that should be listed here.

Major issues

12/2007: overaggressive and inappropriate marketing

How we fell short: As part of an effort to gain publicity, GiveWell's staff (Holden and Elie) posted comments on many blogs that did not give adequate disclosure of our identities (though we did use our real first names); in a smaller number of cases, we posted comments and sent emails that deliberately concealed our identities. Our actions were wrong and rightly damaged GiveWell's reputation. More detail is available via the page for the board meeting that we held in response.

Given the nature of our work, it is essential that we hold ourselves to the highest standards of transparency in everything we do. Our poor judgment caused many people who had not previously encountered GiveWell to become extremely hostile to it.

What we are doing to improve: We issued a full public disclosure and apology, and directly notified all existing GiveWell donors of the incident. We held a Board meeting and handed out penalties that were publicly disclosed, along with the audio of the meeting. We increased the Board's degree of oversight over staff, particularly with regard to public communications.

6/2007: poorly constructed "causes" led to suboptimal grant allocation

How we fell short: For our first year of research, we grouped charities into causes ("Saving lives," "Global poverty," etc. - see the purple bar at the top of this page) based on the idea that charities within one cause could be decided on by rough but consistent metrics: for example, we had planned to decide Cause 1 (saving lives in Africa) largely on the basis of estimating the “cost per life saved” for each applicant. The extremely disparate nature of different charities’ activities meant that there were major limits to this type of analysis (we had anticipated some limits, but we encountered more).

Because of our commitment to make one grant per cause and our overly rigid and narrow definitions of "causes," we feel that we allocated our grant money suboptimally. For example, all Board members agreed that we had high confidence in two of our Cause 1 (saving lives) applicants, but very low confidence in all of our Cause 2 (global poverty) applicants. Yet we had to give equal size grants to the top applicant in each cause (and give nothing to the 2nd-place applicant in Cause 1).

What we are doing to improve: We plan to research fewer, broader causes in the future, giving ourselves more flexibility to grant the organizations that appeal to us most. We plan to explore broad sets of charities that intersect in terms of the people they serve and the research needed to understand them, rather than narrower causes based on the goal of an “apples to apples” comparison using consistent metrics.

6/2007-11/2007: set overly ambitious deadlines for completing our research

How we fell short: In July 2007, when we mailed out grant applications to charities, we told them that we would award grants by December 2007. We did not finish all necessary research by December 2007, and thus only paid out 3 of our planned 5 grants by the agreed upon date. We paid the last two grants in early March of 2008.

What we are doing to improve: Setting deadlines was particularly difficult for our first year, as we had no previous experience with grantmaking. With the first year behind us, we have a better sense of how time-consuming research is, and in the future we will set more conservative expectations.

6/2007-5/2008: research process relied excessively on open-ended grant applications

How we fell short: In our first year, we focused our time and effort overwhelmingly on getting information from applicants, as opposed to from academic and other independent literature. Applicants found this process extremely time-intensive and burdensome, particularly given the size of the grants. We also found that much of the information we found essential in making informed decisions was not submitted through grant applications (rather, we found it through independent research).

What we are doing to improve: We are modifying our research process. In the future, we will start by searching independent research to identify particularly promising approaches; we will send out applications (for our grants and recommendations) later in the process, and while we will still invite charities to make their case in whatever way they deem appropriate, we will treat the application primarily as a way of getting specific information about specific organizations rather than as the starting point of our research. We believe that doing so will improve our ability to collect information, and lower the burden on applicants.

Smaller issues

2/2008-5/2008: premature hiring

How we fell short: In an attempt to increase our research capacity, we hired our strongest volunteer in January, but terminated the relationship at the end of May. We have mutually agreed that, at this stage of our development, we can't provide the training and management necessary for someone of his skill set to add significant value.

What we are doing to improve: We have determined that for the time being, capacity building is not a primary goal; the overhead and risk involved in finding and incorporating a new hire would interfere too heavily with higher priorities at this point. This year, we are focused on testing and demonstrating that demand for rigorous research into giving options exists. If and when we answer that question in the affirmative, the next major priority will be building capacity.

12/2007-present: website not sufficiently engaging, has generated too little substantive feedback

How we fell short: We believe that our current reviews, while thorough, are overly dense and difficult to engage with. Feedback from our supporters has often included this theme. In addition, we have received relatively little critical engagement with our analysis from those not directly involved in the project. Our discussion forum has gone practically unused, despite being linked from the top of every review page. We have received many emails offering general support or asking us to consider a particular charity, but the number of people who have critiqued the content of our reviews – through email, survey, discussion forum, or our blog – remains very low. Our site is designed so that people can take quick action based on our recommendations, or dig into the details of our reasoning; we suspect that many people have opted for the former, and very few have opted for the latter.

What we are doing to improve: We believe that we can do a much better job presenting the most relevant and interesting information up front rather than sticking to the highly systematic approach we’ve used to date (which tends to present information in a fixed, consistent order rather than in order of importance and interest). Our research summary, and the documents linked from the top of it, represent a preliminary attempt to do so. We believe that the more conversations we have with supporters, identifying the information that stands out to them, the better we will be able to design our reviews in a way that encourages engagement and interest. It is important to note that improving the website is not a major priority over the next year, as we plan to focus more on personal fundraising (see our plan for more).

12/2007-5/2008: research process should have incorporated more and earlier site visits

How we fell short: We undervalued the role of the site visit during our first year; speaking directly with staff gives us the opportunity to get a clear picture of how an organization views itself, and therefore what sorts of information we should seek to get a picture of whether its approach works as intended. Applicants encouraged us to put more of our time into personal visits, so that we could “get to know” organizations rather than thrusting pre-defined questions on them.

What we are doing to improve: We have found site visits extremely useful in forming a picture of an organization and determining the most relevant questions for it. In the future, we will be modifying the structure of our research process, conducting site visits before we ask applicants to submit detailed organization-specific applications. We believe that doing so will lead to better-constructed applications that give us more information while imposing less burden on applicants.