Some initiatives that have been improving the municipal official public gazettes regarding open data
Image: Open Knowledge Brasil
The "Diário Oficial", or "Public Gazette", is the channel used by the public administration to publish its official acts. It is through these documents that the population learns whether or not a law has come into force, or if a bidding process is already available. In other words, they are extremely important documents.
In Brazil, we have official gazettes at the federal, state, and municipal levels. A very interesting project on this theme that got to know in 2021 is Querido Diário, an initiative by Open Knowledge Brazil. The project's objective is to create a repository of the official gazettes of all of Brazil's municipalities in an open data format.
Currently, official journals are generally published in PDF format, a format which is not considered open.
But what are open data and open formats? And why are these open formats necessary?
According to Open Knowledge Brazil itself:
"Open data is data that can be freely used, reused and redistributed by anyone - subject, at most, to the requirement of source attribution and sharing by the same rules."
One of the very important characteristics of this concept is the ability of the information to be analyzed by machines or computers. In the example of the official gazettes themselves, the daily amount of data produced is quite voluminous. And to be able to create more relevant analyses of this large volume of data, it is necessary that they are automated. Creating this automation of analysis with PDF documents, which can be full of information such as spreadsheets in image format, is more difficult than the analysis of documents in completely text formats such as TXT or XML.
So the big task for Querido Diário is to be able to make the gazettes of the municipalities, originally in PDF, available in open formats like the examples above.
In addition, they are also creating a guide so that the municipal governments themselves can adapt the publication of their official gazettes directly into open formats, avoiding the need to translate information from one format to another.
In this line of thought, Kurytiba Metropole contacted Curitiba's Open Data Technical Committee, which is installed within the Municipal Innovation Council, in order to understand the current situation of the city's official gazettes and then propose improvements.
We received an invitation to join the Open Data Technical Committee and today we are active members of it. Our first action was to search for information about the city's official gazettes.
Like virtually all other Brazilian municipalities, Curitiba publishes its official gazettes in PDF format, a format not ideal for automated analysis.
We discovered when contacting the Municipality that in addition to the gazettes in PDF format, a continuous file (all old and current gazettes in the same file instead of separate files) is also published in CSV format with much of the data from the official gazettes (but not all data).
Therefore, we started a process of dialogue with the municipality to evaluate the possibility of publishing the official journals in a more open format (as recommended by Open Knowledge itself, we opted for the XML format).
The City was very receptive to the idea, and we are now monitoring the process to implement this change. By doing this in the original source of the official gazettes, Curitiba would be a pioneer in open data in this type of document in Brazil. But we also understand the challenges of changing an already consolidated system.
We will continue this monitoring and hope to have more news soon on the development of this initiative.
In parallel, another recent news is that the Querido Diário has included Curitiba, now in May 2022, in its database, allowing access to the official gazettes through its site in original format (PDF) and in text (TXT).
This is not yet the ideal solution, since the TXT of the gazettes is created from the PDF, which can generate errors and omission of some information. But it is already an important next step in maximizing the openness of this fundamental piece of data for the understanding of the city.
And finally, how can this data be used once it is open?
Open Knowledge published a news article with several examples of use cases of data from the official gazettes through the Querido Diário platform in 2021. We cite here one of the examples, related to an analysis of what the official journals say about the LGPD (general data protection law) in 16 Brazilian municipalities. It was possible to quickly create a snapshot of the use of the LGPD through the official acts published in the gazettes. And from this overview, delve further into some issues such as contractual clauses, accountability, governance, appointment of incumbents, alignment with transparency, and capacity building.
This is just one example of the main goal of these initiatives, to be able to have the official public data in an open format and perform analyses for monitoring, understanding, and improving the official public acts.
We will continue this development which is fundamental for a transparent, fair and democratic society!
Kurytiba Metropole Team