Siri is a new Silicon Valley start-up that attempts to change to the way people use the internet. I joined Siri in Sept. 2008, but I was unable to talk about it until this week. Siri (previously known as stealth-company.com) is an SRI spin-off company armed with $8.5M VC funding. The company inherits  technology innovations resulted from many years of AI research (e.g., the DARPA-funded CALO project).

I am quite excited by a recent sneak preview.

The Web, the Semantic Web, the Web 2.0, and Social Networks are all manifestations of the success of the link. The Hypertext Conference provides the forum for all research concerning links: their semantics, their presentation, the applications they have been put to, the knowledge that can be derived from their analysis, and their effect on society.

Main themes in HYPERTEXT 1996 included:

• Spatial Hypertexts
• Autonomous Hypertext Systems and Link Discovery
• Hypertext Rhetoric and Criticism
• Models of Hypermedia Design and Evaluation
• Open Hypermedia
• Navigation in the World-Wide Web
• Systems and Infrastructure
• Extending the World-Wide Web

With many of the above questions, now answered, researchers are moving towards the more "social aspects". HYPERTEXT 2008 themes included:

• Information linking: new models and techniques for interacting with information, automating the "trailblazer"
• Social linking: link inference, analysis and modeling, similarity and retrieval, applications
• Hypertext, culture, and communication
• Applications of hypertext

Submission deadline is February 2009. If you are interested in this area, please consider participating.

Political Streams is an application which mines social media content in real time for political discussion. It surfaces the news articles and documents that are being discussed as well as the people and places that appear in those articles. In addition, it provides related information for any news article, weblog post, person or place. This related information gives a broader context, allowing the user to understand how both the mainstream and social media are discussing an issue, person or place.  

It is this last part that makes this tool interesting.

Very impressive and promising start. Clearly designed to "scale across verticals", this is a result of work by some very smart researchers at Live Labs. I look forward to many of these "yet to be uncovered" verticals. A "200 OK" from social streams is keenly awaited.

Note that crawling blogs is an interesting problem:

• Recency is critical
• Ping servers are available, albeit with incomplete coverage

Crawling blogs is also highly resource intensive:

• Network latency
• Disk access/write latency

How do your numbers look?

The social and community driven aspects of our digital lives continue to rapidly increase, resulting in transformative behaviours and, significantly, publishing and distributing huge amounts of fascinating data. The International Conference on Weblogs and Social Media will meet once more in 2009 to discuss the latest research analyzing and leveraging this resource. As with previous meetings, we will bring together a wide range of researchers and industry practitioners from many disciplines providing a unique opportunity for sharing ideas and collaboration in this space.

John Kleinberg is one of the invited speakers. I wasn’t aware of the "Rebel King" anagram:

Prof. Kleinberg needs no introduction, but how so apt that the above piece of fascinating data is courtesy *social media*.

I recently came across Stephen Baker’s book via the Wall Street Review. This note on splogs got me interested:

A splog, though unreadable, is seeded with words that will attract Google ads. A computer-user may be annoyed at finding himself staring at a screen full of gibberish but click on an ad anyway, allowing the robot blogger to harvest revenue. This sleight of hand has the Numerati hard at work getting their software to distinguish between a blog and a splog. Mr. Baker gives a helpful sketch of the math involved, each blog reduced to a vector in a space of several dozen dimensions.

The problem of splogs, is one case study, through which Baker shares the positive side of the "Numeratis". So what/who is a Numerati anyway? According to Baker:

They’re members of a global elite, and are busy analyzing our every move. They’re rummaging through mountains of data, looking for patterns of our behavior so that they can predict what we might want to buy, who we’re likely to vote for, what job we’d do better than our colleagues. Some are even matching us with potential lovers…

Baker, through his book, uncovers the "numerati cult", who they are, the positives, negatives, and the unknown. Overall, his attempt is to share what these Numeratis mean to, well, a non-Numerati. Yahoo!, Google, and IBM appear to feature prominently, so do many Numeratis.

Elsewhere, both positives and negatives highlighted:

The Corner Office:

The "Numerati" are an evolving class of quant-humping, algorithm experts who will be playing an enormous role in shaping our society, our economy and our lives. They are the types who founded Google and Yahoo but they are going beyond simple searching to manipulating and massaging the tremendous mass of data that we generate from Web clicks and cell phones.

Sentimine:

I have already ordered my copy…How could I resist when we’re mining the blogoisphere for sentiment and about to test our own home-grown splog detector?

Bacon Rebellion:

…“The Numerati,” a class of math experts who quietly orchestrate the massaging of the zillions of bits of data about us. We generate the stuff every time we use our cell phones or search Google, use a grocery loyalty card or whisk through a toll booth using a Smarttag.

ThinkOR:

I think it is great that operations research is getting some publicity with The Numerati. However, there can be such a thing as a bad publicity. Is it just me or does it seem to everybody (OR folks) that this book is casting us in a rather negative light?

Michael Trick:

This is a book primarily about what I would call data mining and clustering, so there are wide swathes of the “numerati” field that are not covered.  But for a popular look on how our mathematics is used to characterize and predict human behavior, The Numerati is an extremely interesting book.

I hope to see this book influence, and promote the positives. The target audience are the non-Numerati’s. But still, this has piqued my curiosity, ordered.

..but we are constantly evaluating everything, which can include:
- proposed improvements to segmentation of Chinese queries
- new approaches to fight spam
- techniques for improving how we handle compound Swedish words
- changes to how we handle links and anchortext
- and everything in between

Though spam and the web graph feature prominently, it is interesting to note how "internationalization" features in many of these evaluation examples, reflecting Google’s overall push in this direction. Evaluation is through click-through improvements, and statistically sound relevance metrics.

Evaluation metrics, I think, is one of those areas where academia can greatly influence and impact search. This is one of the more theory centric problems in search, not limited by the lack of large information retrieval data sets. Indeed, the recently concluded SIGIR conference featured many papers in this direction.

Score Standardization for Inter-Collection Comparison of Retrieval Systems [PDF]
W. Webber, A. Moffat and J. Zobel  (University of Melbourne)

The Good and the Bad System: Does the Test Collection Predict Users’ Effectiveness? [PDF]
A. Al-Maskari, M. Sanderson and P. Clough  (University of Sheffield)

Retrieval Sensitivity Under Training Using Different Measures [blog]
B. He, C. Macdonald and I. Ounis  (University of Glasgow)

Evaluation Over Thousands of Queries [PDF]
B. Carterette, V. Pavlu, E. Kanoulas, J. Allan, and J. A. Aslam  (University of Massachusetts Amherst/Northeastern University)

Novelty and Diversity in Information Retrieval Evaluation [PDF]
C. Clarke, M. Kolla, G. Cormack, O. Vechtomova, A. Ashkan, S. Büttcher, and I. MacKinnon  (University of Waterloo)

Relevance Assessment: Are Judges Exchangeable and Does it Matter [PDF]
P. Bailey, N. Craswell, I. Soboroff, P. Thomas, A. de Vries and E. Yilmaz  (NIST/Northeastern University/Microsoft/CWI/CSIRO ICT Centre)

Intuition-Supporting Visualization of User’s Performance Based on Explicit Negative Higher-Order Relevance [link]
H. Keskustalo, K. Jarvelin, A. Pirkola and J. Kekalainen  (University of Tampere)

Elsewhere, comments on the article  –

Seo Dialect:

The rest of the points are things we’ve been hearing from Google for a long time. We know they’re progressing on universal and personalization search efforts, all in their famous intent to create the best user experience.

Webtribution:

Anyone remotely involved in SEO or digital marketing should always take advantage of any information / insight Google opens to the public.

SearchEngineCaffe:

One of my biggest issues with TREC and similar environments is the single focus on relevance … for example, a spam post that is relevant to a topic would be acceptable, even if you would never want to read it in real life. It’s time we move beyond the basics and find ways to tackle the more challenging retrieval quality aspects…

Also, at webmasterworld.

For readers interested in the overall problem of IR evaluation, a paper by Kalervo & Jaana on "IR evaluation methods for retrieving highly relevant documents" offers an excellent introduction.

Update: Site Now Carries this message:

LinkedIn is currently unavailable while we make upgrades to improve our service to you. We’ll return around 12:00am (PT) September 7th, 2008.

We apologize for the inconvenience and appreciate your patience. Thank you for using LinkedIn!

Btw, Akshay Java, and UMBC sounds very familiar. I wonder why.