Data tales – part one (or Ibn Battuta looks for data)

A defining feature of urban poverty alleviation programmes in India in the last decades has been their ever-increasing size, scale and speed of implementation. The government plays a leading role in the planning and execution of these programmes in collaboration with large (often multi-national) private sector consultants. And increasingly vast amounts of digital data are created in the process, data that is almost always beyond the capacity of the government to store, process and analyse.

It’s extremely important to understand the technical and operational aspects of the programmes, whether one wishes to offer constructive criticism or simply evaluate and make sense of their alleged impacts. Exploring in detail ‘how’ a project is executed also throws light on ‘who’ actually implements it, using ‘what’ tools and techniques. One can then also understand what happens to the data generated and who is effectively in charge of it.

One amusing fact, which many researchers of the Indian developmental ecosystem tend to misinterpret, is that when the government says, "we can’t share the data with you" (which they almost always do) – they really mean it (even if they may not be conscious of the real reasons). They can’t share the data primarily because they may not even physically possess it, or even if they do, they are unaware of its existence!

Data hunting…

In 2017, when I had just started working as a consultant for the government of Odisha, one of my tasks was to visit various offices operating under the department of Housing & Urban Development (H&UD) and scoop up whatever GIS data I could find on the slums of Odisha.

I was quite used to all manners of ‘data collection warfare’ that one has to do with government departments in India to get even the most irrelevant bit of information. So going data hunting with the seal of the principal secretary of H&UD should have been a walk in the park. What followed was as amusing as it was data-enriching.

After hopping from cubicle to cubicle in the Bhubaneswar Municipal Corporation (BMC) office, I finally reached a corner desk where a very polite official told me what I was waiting to hear: "Unfortunately, we have no GIS data on slums – or for that matter on anything else. The Bhubaneswar Development Authority may have files of road networks and so on, but they won't give them to you." The last few words were said with a particularly kind smile, which strongly suggested that he was speaking from experience.

Was he an employee of the BMC? Well, yes... and no. He too was a consultant on the payroll of a private company. After some conversation (people without conversation skills should never go data hunting in India) he seemed to remember an old folder that was stored in the desktop computer on the side and which, judging by the coat of dust, had not been switched on in a long time.

As he navigated to the folder, he remembered more: "The RAY GIS data was stored here... then the programme was abandoned and no one used it anymore."

RAY – or Rajiv Awas Yojana – was a promising, pan-Indian scheme of the Ministry of Housing and Poverty Alleviation, which died a premature death when the previous regime lost political power in 2014. However, RAY had created comprehensive guidelines for preparing geospatial databases of slums across India and many slums had indeed been surveyed before the programme shut down.

To my astonishment, I found that the folder contained a whopping 1GB of geospatial data, complete with slum boundaries, streets, land ownership details and slum house footprints for all 436 slums of Bhubaneswar – as well as the detailed household survey results from around 100,000 slum dwellings.

What was this treasure doing lying forgotten inside a dusty desktop? I wasted no time copying all the data to my hard drive. So much for "Unfortunately, we have no data on slums."

It’s not about the money

One of the reasons for the reluctance of concerned government departments to use, store and update geospatial data is that despite the enormous developments in the field of free and open-source GIS software in the last decades, the prevalent belief continues to be that one has to use extremely expensive, proprietary GIS software.

The price of the software also acts as an effective barrier to mass learning and use. This affects the ability of community organisations to take ownership of digital geospatial data of their own settlements and intervene meaningfully in the spatial decision-making process (I shall deal with this aspect in detail in my next blog).

As a result of all the above factors, the whole task of generating GIS data is outsourced to private consultancies, who undertake the geospatial surveys, prepare the databases and store them on their own servers for the duration of the project.

On termination (normal or premature) of the project, the data is (theoretically) ‘handed-over’ to the government, which neither has the systems nor the professional capacity to ‘accept’ it. The data is mostly lost, misplaced, forgotten or tucked away in dust-coated desktops (which is actually the best-case scenario). Such is the fate of gigantic pools of digital data that are prepared as part of large development projects in India.

Share and share alike – or even reshare

Two years later, in a presentation to the newly appointed commissioner of BMC, I showed the same data to him. To my astonishment, he asked if I could share this valuable data (which was his own office’s data) with him as his office had no such data on slums. Of course, by ‘office’ he meant the members of the private sector consultancy that staffed his project management unit.

Data is definitely power, but to have the tables turned so completely one needs not only the power of data but of data obsolescence acting in one’s favour. I ‘reluctantly’ (I couldn’t resist some drama at so ridiculous a moment) agreed to ‘share’ the data on the condition that I would also install the open source QGIS software on the laptops of his consultants and teach them the basics of operating the software. They readily agreed. Although, I must confess that I was very tempted to end the discussion with an “unfortunately, this data cannot be shared”.

I was waiting for them to ask me about the source of the data – they never did.

More on data tales

We invited Antarin Chakrabarty to contribute to this blog series and reflect on his experience working in Jaga Mission, the state of Odisha’s ambitious slum upgrading programme. His particular interest is how well this was served – or not served – by GIS.

• Data tales part two reports on what the author found when he went looking for relevant data. Despite being told by government officials that no such data existed, he discovered vast resources that were not being used
• Data tales part three reveals the (unexpected) technological prowess of the urban poor
• Data tales part four shows how the lack of relevant or accurate data on climate risk and resilience in slums could be resolved by making community-based data collection the norm.
• Data tales part five questions whether the flashy and often unnecessary ways we use computers have detracted from their primary function of reliving humans of boring, repetitive tasks.