{"id":955,"date":"2018-10-04T08:39:24","date_gmt":"2018-10-04T06:39:24","guid":{"rendered":"http:\/\/muzeum.netfolio.hu\/?p=955"},"modified":"2025-06-05T13:21:18","modified_gmt":"2025-06-05T11:21:18","slug":"befejezodott-20-evfolyamnyi-arjegyzolap-digitalizalasa","status":"publish","type":"post","link":"https:\/\/tozsdemuzeum.hu\/en\/2018\/10\/04\/befejezodott-20-evfolyamnyi-arjegyzolap-digitalizalasa\/","title":{"rendered":"Digitisation of 20 vintages of Price Quotation Journals completed"},"content":{"rendered":"

We are pleased to announce that, after almost two years of work, the digitisation of the Budapest Commodity and Stock Exchange Price Quotation Journals, published between 1894 and 1913, was completed in October 2018.<\/p>\n

The beginnings<\/h2>\n

The idea to process old stock market data first occurred to M\u00e1rton Radnai, CEO of Ramasoft Data Services and Information Technology Ltd., almost 10 years ago when he learned that past US stock market data is now available on a daily basis for almost 200 years. It was then that the idea of processing this data for Hungary was conceived. But at the time, neither market nor public funding for the project could be found.<\/p>\n

However, over the past 10 years, the cost of digitising old journals has fallen significantly as Arcanum Database Ltd.<\/a> digitised one of the stock market price sources, the Budapest Public Notices<\/a> and made available at its own expense the Arcanum Digital Library<\/a>. So two years ago, M\u00e1rton Radnai decided to start the project with his own funding.<\/p>\n

Find resources<\/h2>\n

The project started with a search for available data sources. Until the First World War, the prices of the old stock exchange were published in three journals: the Budapest Price and Stock Exchange Price Quotation Journal, published almost uninterruptedly between 1864 and 1948, the Budapest Gazette (the official state gazette of the time) and Pester Lloyd. Of the three publications, the Budapest Gazette provided only partial data (no prices for deals, only closing prices) and Pester Lloyd was in German. After the First World War, the data were published for a while only in the Quotation Journal, and then again in the Budapest Gazette, but only in a highly abstracted form. It became clear that the optimal solution would be to digitise the Quotation Journal.<\/p>\n

\"\"
Some volumes of the price lists are still in their original binding.<\/figcaption><\/figure>\n

The number of copies of the Quotation Journals was not very high, so very few of them have survived, only a few Hungarian libraries had them, and some volumes were available in the Austrian national library. Of these Szab\u00f3 Ervin Library of Budapest<\/a>\u00a0The Budapest Collection (where the years 1873-1913 were available) has agreed to make its collection available for digitisation. The digitisation was carried out by Arcanum Database Ltd. Negotiations between the three partners were finalised by April 2017, and digitisation began.<\/p>\n

Photography<\/h2>\n

Since the printing technology for the 1873 to 1893 years had not yet have the quality that would have allowed for later computer processing of the data, and since the data content of the price lists for this period was essentially identical to the already digitised Budapest Gazette, the project sponsor Ramasoft decided to digitise the 1894 to 1913 period in order to be cost-effective.<\/p>\n

\"\"
The first step was to remove the previous bandage, which was done with this cutting machine.<\/figcaption><\/figure>\n

This was done using two techniques: the 1894-1904 cohorts were photographed using a photographic machine. However, for the later years (as the price sheet doubled in size), the images were captured using a so-called map scanner.<\/p>\n

\"\"
The lighting was then installed.<\/figcaption><\/figure>\n

Pre-processing of images<\/h2>\n

After the exposure, the images were handed over to Ramasoft by Arcanum. In order to improve the quality of the subsequent optical character recognition, Ramasoft developed a special software that performed two functions: firstly, it automatically straightened and trapezoidally corrected the price tables, so that the rows of the table were only horizontal, which is a prerequisite for efficient character recognition. On the other hand, he transformed the tables to the same pixel position every day in order to set a template in the optical recognition software for processing them.<\/p>\n

\"\"
The original image<\/figcaption><\/figure>\n

However, the character recognition result was not yet satisfactory, as the recognition software could not separate the rows of the tables. Therefore, in a second step, the dividing lines of the tables were drawn. The tables could then be recognised with sufficient quality.<\/p>\n

\"\"
The corrected image<\/figcaption><\/figure>\n

Optical character recognition<\/h2>\n

For optical character recognition, Ramasoft used the Abbyy Finereader application. This was done in two phases: in the first phase, the straightened and trapezoid-corrected images were converted into a so-called two-layer PDF, which allows them to be read later.<\/p>\n

In the second phase, the version including the drawn table lines was character-recognised and the data exported to Microsoft Excel.<\/p>\n

\"\"
Recognising the corrected image in OCR software<\/figcaption><\/figure>\n

 <\/p>\n

\"\"
Raw data exported to Excel<\/figcaption><\/figure>\n

Data verification and correction<\/h2>\n

The raw excel output was then sorted into a database. To do this, we built a securities data list and a dictionary that matched the names of the securities in the newspaper with the names in the data list. A major difficulty was that in many cases a catcount was used to name securities, which made this matching difficult. In addition, the exchange rate data had to be cleaned and formatted (for example, the quotations of the time did not have decimal separators).<\/p>\n

From the data thus paired and cleaned, another excel file was created, in which on the one hand automatic control rules were checked, and on the other hand this file was given to the proofreaders to compare with the original documents and to correct remaining errors manually.<\/p>\n

\"\"
Paired and corrected data in Excel<\/figcaption><\/figure>\n

Exporting data to a database<\/h2>\n

The last step was to export the corrected data to an SQL database.<\/p>\n

 <\/p>\n

 <\/p>","protected":false},"excerpt":{"rendered":"

\u00d6r\u00f6mmel jelentj\u00fck be, hogy csaknem k\u00e9t \u00e9vi munka ut\u00e1n 2018 okt\u00f3ber\u00e9ben befejez\u0151d\u00f6tt az 1894 \u00e9s 1913 k\u00f6z\u00f6tt megjelent Budapesti \u00e1ru- \u00e9s \u00c9rt\u00e9kt\u0151zsde \u00c1rjegyz\u0151 Lapjainak a digitaliz\u00e1l\u00e1sa. A kezdetek Az r\u00e9gi t\u0151zsdei adatok feldolgoz\u00e1s\u00e1nak \u00f6tlete el\u0151sz\u00f6r csaknem 10 \u00e9ve mer\u00fclt fel Radnai M\u00e1rtonban a Ramasoft Adatszolg\u00e1ltat\u00f3 \u00e9s Informatikai Zrt. vez\u00e9rigazgat\u00f3j\u00e1ban, amikor megtudta, hogy a m\u00faltbeli amerikai […]<\/p>\n","protected":false},"author":1,"featured_media":771,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"_monsterinsights_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0,"footnotes":""},"categories":[1],"tags":[],"class_list":["post-955","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-egyeb"],"acf":[],"_links":{"self":[{"href":"https:\/\/tozsdemuzeum.hu\/en\/wp-json\/wp\/v2\/posts\/955","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/tozsdemuzeum.hu\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/tozsdemuzeum.hu\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/tozsdemuzeum.hu\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/tozsdemuzeum.hu\/en\/wp-json\/wp\/v2\/comments?post=955"}],"version-history":[{"count":16,"href":"https:\/\/tozsdemuzeum.hu\/en\/wp-json\/wp\/v2\/posts\/955\/revisions"}],"predecessor-version":[{"id":12079,"href":"https:\/\/tozsdemuzeum.hu\/en\/wp-json\/wp\/v2\/posts\/955\/revisions\/12079"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/tozsdemuzeum.hu\/en\/wp-json\/wp\/v2\/media\/771"}],"wp:attachment":[{"href":"https:\/\/tozsdemuzeum.hu\/en\/wp-json\/wp\/v2\/media?parent=955"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/tozsdemuzeum.hu\/en\/wp-json\/wp\/v2\/categories?post=955"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/tozsdemuzeum.hu\/en\/wp-json\/wp\/v2\/tags?post=955"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}