WeKnowIt Image Recognizer

This application allows mobile phones users to quickly discover the location and name of photographed objects.
It was developed by Software Mind S.A., National Technical University of Athens and Yahoo! Research, Barcelona.

Contact: Rafał Janik

Project

This work is funded by the WeKnowIt project.
WeKnowIt (ICT-215453) is an Integrated Project (IP) of the European Union's 7th Framework Programme: Information and Communication Technologies (ICT).

Demo

Requirements

Application can be run on any Android 2.0+ phone. It was tested on HTC Hero 2 mobile.

In order to make it possible to analyze images from SD card an additional component has to be installed: OI File Manager. It can be downloaded from Open Intents web page.

Download

The application is available here.

Technologies

This application uses two services that are part of the weknowit project:

WP2_VisualAnalysis developed by NTUA/CERTH-ITI,
WP3_GetPOIsAt developed by Yahoo! Research, Barcelona.

The code was written using Android SDK.

Description

The aim of the application is to provide user with the detailed information on the location and name of a POI that she photographed. The application uses services developed by WeKnowIt project partners in order to:

recognize object (POI - Point On Interest) on picture,
determine its geolocation,
determine tags associated with this POI,

Image recognition is provided using the VIRaL (Visual Image Retrieval and Localization) application which is a content-based image search engine developed by NTUA (National Technical University of Athens) and currently contains more than 1 mln Flickr® images from 30 European cities. POIs are retrieved from a list of geo-coded Wikipedia pages.

Additionally application contacts Wikipedia and gathers information on the recognized POI and Google Maps to get POI localization on the map.

Details

The following services are used by the WeKnowIt Imager Recognizer application:

WKI System - which provides an easy to use REST API and collects results returned by other servers,
WP2_VisualAnalysis - which determines the geolocation of the query image, suggests tags to be added to it and identifies visually similar photos within seconds, without giving any text!
WP3_getPOIsAt - which returns a list of close-by POIs for a given geolocation. The POIs correspond to geo-coded Wikipedia pages.
Wikipedia - which returns description of POI.
Google Maps - which returns POI map localization.

1. First user takes a picture and uploads it to the WKI System.

2. WKI System passes image to the WP2_VisualAnalysis service which responds with its geolocation and Flickr® tags.

3. Received geolocation is passed to WP3_getPOIsAt which returns name of POI associated with this geolocation.

4. Description of POI is retrieved from Wikipedia.

5. Finally the collected data are send to users mobile and presented to user.

6. User can find localization of a POI on the Google Maps.