Interactive Book: Lisbon by Pessoa

The link has been present for some time in the menu on the left, but the project is finally mature enough to be presented. In 1925, Fernando Pessoa, the famous Portuguese poet, wrote a touist guide about the city he almost never left: Lisbon. A text with no poetic intention, written directly in English (the complete title is Lisbon, what the tourist should see), to tell the world about the marvels of his beloved city. They were quite preserved in the XX^th century, and modulo renaming, most of the monuments cited and their descriptions have not changed since. The guide was therefore translated to many languages after being discovered in the end of the 90s in the author's manuscript "trunk".

The text was first published in a bilingual English-Portuguese version by Livros Horizonte. Unfortunately, this edition contains the original text without editor 's notes or index, and with only a map of the city in 1929, rather difficult to read. Therefore, it can hardly be used to find information while visiting Lisbon. However the text was reedited this year in English by a British publisher, Shearsman, with some more content. They updated the names of the places and persons into their modern style, and added some photos of the city from postcards from the 20s.

To make this guide even more useful for the tourist, I've created an interactive version of Lisbon, what the tourist should see, with a Google map, and some photos taken during a lovely week spent in the city, as well as some found on the Wikipedia or Flickr.

I scanned the Livros Horizonte version of the book which I had just brought back from Lisbon, performed optical character recognition with SimpleOcr (not very reliable, but free...), to get the complete text which is now freely available (Pessoa has been dead for more than 70 years) at:

http://lisbon.pessoa.free.fr

Places and streets cited in the guide were then localized on a map of Lisbon, to get a geographic visualization of the book, where Pessoa gives in fact three circuits - the first one, in blue, is quite long, the other two, green and red, are given for the tourist who"can stay one day more". The book ends with a description of Portuguese journals at that time, then details of some villages in the area. The main (blue) itinerary, which starts from the sea, requires a car. In fact, as it is impossible to make all visits in one day, it can be split into many parts that can be visited on foot or with public transportation. But be careful, in this case, follow the map instead of the ordering of the visits in the book, as the path described there is absolutely not a solution of the TSP! This choice is not random either, as Pessoa distributed the most important visits (the Baixa district, the Alfama, the Castle of Saint George, the Hieronymites Monastery, the Tower of Belém, etc) uniformly along his text.

A Google map is quite nice, but not so useful if you travel without internet! By the way if you're looking for an internet connection in Lisbon try Rua da Madalena. The map is also available in a printer-friendly version, with a number associated to each place (don't worry if the page takes some time to load, usually more thant 10 seconds for me ;)). To get the labels of those numbers, sorted as they appear in the text, go to the bottom of this printer-friendly version of the text.

If you have access to the interactive version though, you get much more information. For many places, there is a link to its Wikipedia page, or even its official website (with opening hours, for museums).

This supplementary information, which transforms this text into an interactive book, has not been added directly to the original text. In fact I created a PHP+MySQL+Javascript system to automatically insert in the text this information which is stored in databases. The picture below tries to explain the principle. Besides the text, there are 3 databases: the blue one which stores locations in the text of occurrences of the places, the orange one stores the places, and the purple one stores coordinates. Now let's explain the arrows below. For a given set of coordinates on the map, stored in the purple database, there may be one or many interesting things to see (for example on Praça do Comércio there is also an equestrian statue of King José I). Each of these things to see has a file in the orange database, which gives its name and description, sometimes a photo as well. Note that if you want the website to give you information on places in another language, you just have to translate this database and not the whole site! Finally, to know where all these interesting things appear in the text by Pessoa, the position of the characters where they appear is stored in the blue database. It is then possible that one of them appears at different places in the text, like Praça do Comércio below. If the original text is modified (translated, for example), then this blue database has to be changed too.


To finish the project I still have to complete the orange database (I've currently done more than one third). However you can already access everything added so far, especially the text illustrated with photos here. And of course the Google map, which is the basic element of this mashup (contact-me to get the sources if you have a similar project of interactive book) on a theme by Pessoa.

So you can start planning your one week (or more) trip to Lisbon in good company: the one - at least - of some elements, printed or downloaded, from the site...




This post was originally published in French: Livre interactif : Lisbonne par Pessoa.

Cooking for nerds: ingredient polyhedron and convex hull

Even if I'm not an expert in molecular gastronomy, I'm often very impressed by transformations of form, color or texture that happen when I cook. It's so nice the mutations of those choux au Comté in the oven, or how beaten egg whites can turn into crisp meringues on my radiator. Don't worry, I won't talk about chemistry and how those reactions work, but just to what extent it can work.

Recipes are so accurate: you get a list of ingredients with exact quantities, and how to use them, but no warranty on what happens if you don't exactly respect the quantities. That's why I will define a tool to represent ingredient quantity robustness in a recipe in this post: the ingredient polyhedron. And a method to compute it from many receipes of the meal you want to cook, found on the web for example. My example will be crêpes, our French flat pancakes.

This dessert is done with roughly 3 ingredients (and of course butter for the pan, but we will just speak about ingredients of the batter), we will thus get a very nice 3D picture. So: eggs, flour, milk, those ones appear in all of the 19 recipes I've gathered in this OpenOffice spreadsheet file thanks to the following websites: lejus.com, 1001delices.net, recette-crepe.net, goosto.fr, supertoinette.com, recettes.qc.ca and the French reference Marmiton (sorry for my vegan friends). But maybe I'll just start with 2 ingredients to show how the whole thing works. Say we have already decided the number of eggs to use, one for example. We then compute according to all recipes, with a rule of three, the quantity x of milk and y of flour that have to be added (I translated everything to grams for simplicity). Those coordinates can then be plotted on a graph, to get the following dots:

From the lower left to the upper right, the number of eggs in the recipes decreases (as there is more and more flour and milk). On the upper left corner we have lots of flour, and on the lower right corner, more milk. And what is this kind of elastic band which sticks around the dots? It's some kind of safety area: any point within this area should correspond to ingredient quantities that works for the recipe. Well, at least that's what I hope: any point of the segment between any two points that work should work too, send me your counterexamples if you do not agree. Anyway, this area is called the convex hull of the point set, and there are many algorithms to compute it automatically. So of course to avoid taking risks you may want to target the middle of the convex hull. Notice that 3 recipes with the same main ingredient quantities correspond to a quite central dot (half a liter of milk and 250 grams of flour for 3 eggs).

The convex hull also shows the robustness of the recipe according to each parameter, that is how accurate you have to be when measuring each ingredient. Note how narrow the convex hull is (it would be even more if I had chosen the same vertical and horizontal scale). This means that depending on the recipe, the quantity of eggs may vary a lot, but not the proportion milk/flour. We can plot for each recipe the difference between the ratio of two ingredients, and the average ratio for those two ingredients:

If you do the average of the absolute value of those deviations, you get: 16% for the milk/flour ratio, 28% for flour/eggs, 31% for milk/eggs. The milk/flour ratio varies much less than the other ratios among the recipes, so you have to be more careful in this proportion than when choosing the number of eggs. So we have just illustrated and proven this nice theorem: the recipe of crepes is pretty robust to the variation of the number of eggs.

You can also have fun by showing many desserts which share the same main ingredients on the same graph:
Well, just wait before pouring your "pâte à crêpes" onto waffle iron: you may want to add some baking powder and vegetable oil...

To conclude, let's take a look at the 3D ingredient polyhedron thanks to this very nice applet by Tim Lambert (he even shares the source that I was able to modify to include my crepes points), you can use the mouse to control and move it:


Sorry, but you need Java to see the animation.

Here again what wa see is a convex hull, in 3 dimensions, on dots (x,y,z) where x is the number of eggs, y the quantity of milk and z the quantity of flour. I put the dots by choosing a minimum limit and a maximum one on the number of eggs to get this frustum, such that any cut perpendicular to the x axis (for a constant number of eggs) gives exactly the same convex hull polygon as above. To really use it we should let the user enter the quantity of ingredients he just used: if the dot gets inside the polyhedron, no problem, otherwise... you may try the restaurant tonight!

What does veronising mean?

Well, to get some idea of what veronising is, maybe you should check Jean Veronis's blog. My definition would be "to design and publish on a blog programs or methods able to help analyzing data". Jean has created a whole bunch of useful tools, which work mainly on texts (he is a researcher in natural language processing) or internet corpuses (search engines results for example). Among the most impressive, the Nébuloscope, which makes tag clouds out of words appearing frequently in the results of a search engine request, or the Chronologue, which used to draw the evolution of a keyword use on the internet (it used the "date" function of a search engin which has now disappeared).

Inspired by his impressive results, I've started to analyze data I find interesting myself, and program some little tools to help me do that. I may translate some of my previous posts, here are some topics I've worked on, I put the links to French posts until they are translated to English.

Phylogenetic trees are used to represent the evolution of species, based on the idea that some species close to each other will appear in a same subtree, and a lot of algorithms exist to build them from biology data. But phylogenetic trees are also an excellent mean of visualizing data, and I've tried building the trees of country votes at the Eurovision song contest, French "députés" (our congressmen) according to their proximity of votes (as well as a DNA chip visualization of those votes), and more recently I've been working on building what I call a "tree cloud" from a text, the same idea than a tag cloud except the order of the words is not alphabetical, but they are displayed as leaves of a tree. Until the program is finished, I still rely on tag clouds (with nice colors and a logarithmic scale, pleaaase, not those ugly and unexpressive ones we often find on the internet !). I've tried using them to analyze one's writing style (with instant messaging logs) or speaking style (with the planned version and the pronounced version of a press conference talk by President Sarkozy).
I like doing some search engine statistics, to help spelling, visualize and date the birth of the web, or send massive requests to compare popularity of people or concepts. Those stats analyzes often make critical use of spreadsheet programs, which also helped me to track the evolution of a petition, which gave me a glance on the time of the day people connect to the internet depending on their job (students, teachers, engineers...). I could also get nice synthesis pictures of French polls before the first round of the presidential election, in 2002 and 2007. I'm very interested in informative and original visualizations, like Voronoi diagrams (for McDonald's restaurants in Paris) or metro map views (building them from a genuine metro map is a GI-complete problem).

I have also analyzed a blog meme last year, the "Z-list", which in France appeared as "la F-list". Even if I did not publish my data on the "Z-list", I still have the files, as well as the "infection tree", on my computer somewhere. This year I've created a little utility, the "CaptuCourbe", to put data from the picture of a curve into a spreadsheet file (some "unscan" programs do this but they are quite complicated to use, or expensive), which helps comparing the evolution of a buzz on many buzz tracking systems (Google Trends, Technorati, site stats systems...). Currently the program is in French only, but Jean motivated me to translate it to English, which will soon be done.

And you will never guess the topic of my most visited blog post, which I'm not the most proud of: I had noticed a bug on some French TV channel website which gave access to the channel live on the internet. It lasted about 3 days, but since then Google sends me all people who want to watch "M6" on the web. I've put links to other French channels which can be viewed free anyway, to avoid frustration.

See you soon for some new computer-powered experimentations!