Tools used in the making of facebook

Facebook is the #1 social-networking site in the world now. Did you wonder what are the tools that made facebook so successful. Beside people and data, which are the key ingredient that help to make facebook so yummy, there lies all those utensils that make it happen. Today, I shall focus on some tools that are use  for Facebook. Most of the description here are extracted from facebook developer site or developer wiki sites.


Scribe - An open source server for aggregating log data streamed in real time from a large number of servers. It is designed to be scalable, extensible without client-side modification, and robust to failure of the network or any specific machine.

Scribe server running on every node in the system will aggregate messages and send them to a central scribe server. If the central scribe server isn’t available the local scribe server writes the messages to a file on local disk, and sends them when the central server recovers. The central server can write the messages to the files that are their final destination, typically on an nfs filer or a distributed filesystem, or it could also send the messages to another layer of scribe servers.

Clients log entries consists of two strings- a category and a message.

Facebook and Twitter is currently using scribe.

Source code at http://github.com/facebook/scribe



Thrift s a software framework(libraries) for scalable cross-language services development. It combines a powerful software stack with a code generation engine to build services that work efficiently and seamlessly between C++, Java, Python, PHP, and Ruby. Thrift allows you to define data types and service interfaces in a simple definition file. Taking that file as input, the compiler generates code to be used to easily build RPC clients and servers that communicate seamlessly across programming languages.

http://wiki.apache.org/thrift/FrontPage


OpenLink Data Spaces (ODS) is a Distributed Collaborative Web Application Platform, Social Network, and Content Management System for creating presence in the semantic web via Data Spaces derived from Weblogs, Wikis, Feed Aggregators, Photo Galleries, Shared Bookmarks, Discussion Forums and more.


Data Spaces are a new database-management technology frontier that deals with the virtualization of heterogeneous data and data sources via a plethora of data-access protocols.

As Unified Data Stores, Data Spaces also provide solid foundation for the creation, processing and dissemination of knowledge, making them a natural foundation platform for the emerging Data-Web (Semantic Web, Layer 1).
 
Data Spaces also provide a cost-effective route for generating Semantic Web Presence from Web 2.0 and traditional Web data-sources, by delivering an atomic data container for RDF Instance Data derived from data hosted in Blogs, Wikis, Shared Bookmark Services, Discussion Forums, Web File Servers, Photo Galleries, etc.
More info please refer to:-

http://virtuoso.openlinksw.com/dataspace/dav/wiki/Main/Ods

Facebook Markup Language is a markup language that is used to customize the "look and feel" of applications that developers create.

It is a variant  of HTML with some elements removed. This allows Facebook Application developers to customise the "look and feel" of their applications, and how to encode content so that Facebook's servers can read and publish it.

FBML set by any application is cached by Facebook until a subsequent API call replaces it. Facebook also offers specialised Facebook Javascript (FBJS) and library.

FBML Tags

• Social data tags
• Sanitization tags
• Design tags
• Component tags
• Control tags

Facebook Platform wiki site:
http://en.wikipedia.org/wiki/Facebook_Platform


FB Development Tools
Facebook Application Programming Interface (API)
Facebook Query Language (FQL)
Facebook Markup Language (FBML)
Facebook JavaScript (FBJS)
Facebook Connect

FB Architecture - Brief

One of the most, if not the most often used social networking site, facebook. Today, I will try to summarize the Facebook(FB) architecture of what I have understood from Aditya(Dir. Engineering, Facebook) talk in Dec 08. (Note: this is my intepretation of the architecture look from the talk.)

Basically FB utilized the LAMP framework, but the modified version, LAMP stands for Linux, Apache, Mysql and PHP/Perl. The wise people at FB has modified and enhance most of the LAMP components, especially Mysql & PHP while including additional services to create FB. Web 2..0 tools like the Scribe, Thrift & ODS  were employed in developing the FB. A giant hash memory table or Memcache also helped to create a fast responsive FB. Services are like, newsfeed,Adserver, Search, network selector,CSS Parser, ShareScrapper, mobile  and blogfeed make up the complete FB architecture.

A notable mentioned in FB design is 'memcache'. Why memcache?. Its is faster and of high performance, alleviate the load on databases, cache serialized PHP data structures and multi-get to retrieve data. FB has modified memcache to run over UDP reducing the overheads and the persistent connection of TCP.

One key note that data can be slightly delay in facebook and need not be accurately realtime, although FB will try to be consistent. Data is also sorted in recentcy layout for optimizations. Data of the form key-value pair is evenly distributed across multiple DB instances and used LB. This allow FB to scale fast and do not require replications. The query in FB application is often a simple query type and they do not allow joins, and need not too.

One thing to always remember is that FB is not just a site about social-usages, but it also allow others to contribute and grow its applications. Developer can write FB apps, and using FBML extensions to easily interface and publish program. Of course the FB application's one developed has to be hosted somewhere other than the FB site. This flexibility allow fast adoption and wide use of FB.