Surveys: getting feedback about our publications

IIED has more than 6,000 publications available to download free of charge from our online library, and while we can see how many times each one is downloaded, we often don't know what happens next. The publications survey was designed to find out more.

We promoted the survey by a mix of methods, resulting in 369 respondents via:

• A pop-up window on the publications section of the IIED website: 77 respondents, with a reasonable level of questionnaire completion 
• Promotion through IIED staff email signatures: 38 respondents, with a high level of completion
• Promotion of the survey through the IIED newsletter (7,500 subscribers) and by email to the IIED 18,000-strong contacts database. The bulk of the respondents (254) were recruited through this method. 

We can assume the survey reached at least 14,400 potential respondents, making the response rate for those contacted by email 1.76 per cent or lower.

What did we learn?

The findings both reinforced what we already suspected and told us something new.

People told us that their main interest areas were climate change, natural resource management, food and agriculture, and biodiversity, which matches up with the higher visitor numbers for those areas of the IIED website.

They told us that the biggest influence on whether they read a publication was the title, the opening summary and the reputation of the author.

This has already fed into training on writing publication abstracts, enhancements to our Publications Library and more work on search engine optimisation.

The most common response to the question of how people had heard about IIED was through: 

• An internet search
• Knowing the URL of our website
• Via our staff
• Twitter, and
• Email newsletters.

This confirmed that IIED's Twitter account and the IIED monthly newsletter are valuable communication channels, not only for sharing information but for reaching new audiences.

We also found that promoting the survey by email signatures and sparingly-used pop-ups was surprisingly effective in leading respondents to a greater commitment to IIED. Sixty-four per cent of those who accessed the survey via the pop-up we installed signed up for our monthly newsletter, for example, and three quarters of those who accessed the survey via an email signature went on to subscribe to the IIED newsletter.

We gleaned a lot of information about what respondents would like more of or to be different – all of which is feeding into our longer term web development plans.

And generally respondents bucked the trend around PDFs, saying they liked to have a PDF for downloading and printing out.

What would we do differently?

This was our second survey and is part of an ongoing evaluation programme. We were aware that the methodology we used would not give the survey results statistical validity and want to do this differently next time. We sent the survey to contacts who knew us already, who then opted to complete the survey; there wasn't an open opportunity for anyone to take part.

This is a common challenge for many organisations trying to survey large and complex target audiences. It does not mean the results have no value but that they are indications only that must be investigated further through focus groups or user testing, rather than being taken as completely representative of the audience.  

Generally speaking, we realised that we needed to do more qualitative evaluation to support our regular monitoring of metrics such as downloads of publications and views of pages of the website, and build our own capacity to do it as effectively as possible.

• Read more about what we learned from our publications survey.