This year web 2.0 summit focus on key point of control

How the clashes between the giants and dwarves, coexisting and fighting for control in the web 2.0 world. Companies consolidating for power and increasing the footprints. The theme for this year summit on the point of control certainly will help the community learn how this controls are working and how it will shape the future.

So how does it help? A map was created.
The  first iteration map that could be found at this link below:-
http://map.web2summit.com

Hope to learn alot more soon.

Afraid of loosing...

To understand the design, sometime we have to go back to some human fundamentals...

I just learn an interesting view on human perspective on loss aversion which I hope to record here. 
Based on given amount and head or tail challenge.


Case 1:
Given $1000

Take a risk:  [ Head get $1000 more, Tail loose and get $0]
Play it safe : [ Get $500 more no matter what ]


Case 2:
Given $2000

Take a risk:  [ Head loose $0 , Tail loose $1000]
Play it safe : [ Loose $500 no matter what ]


It was understood even in human and some primates study, both will more preferably choose to play it safe in case 1, but when the scenario is changed in case 2, more will take the risk option. The interesting point to observe here is that in both cases, playing it safe will make one person still have $1500. However, the relative view of human mind chooses bases on the perception where it is called risk loss aversion

Interestingly, that how our finance/economy is geared up to capitalize on this psychological aspect of human  behaviors.

A quick blog on LTO tapes roadmap and specs ( Photocourtesy of LTO Ultrium owners)

android is learning to run

Just like a baby, after 18 months or so, Android OS has started to show that is able to run. If cupcakes was liken to a baby learning to walk, Froyo will likely be it learning to run. Infact it is already showing some strength against the iphone OS prior froyo release.

Here is the list of milestone to date on android OS.
20080923 - Android 1.0 [1st release]
20090309 - Android 1.1 [2nd release]

20090430 - Android 1.5 [cupcake]
20090915 - Android 1.6 [donut]
20091026 - Android 2.0 [eclair]
20100112 - Android 2.1 [flan]
20100520 - Android 2.2 [froyo]
TBA         - Android 3.0 [gingerbread]


 

*picture courtesy of web resources*

Mathematica & Wolfram Alpha

Been busy quite lately with works piling and planning for some overseas work. I have came across something useful today. It a site called wolfram alpha. They even have cool mathematical and computational tools now ver 7.0 mathematica.

Well wolfram alpha is about making computational mathematical online and it is really cool. wonder if smartphone is allow at school or exam.. I dont have to remember all those differential equations steps if that allow ...

http://www.wolframalpha.com

AHAH - Asynchronous HTML over HTTP

AHAH is a simpler version of AJAX.  AJAX uses the XML file while in AHAH, it uses XHTML
In AHAH you can display external XHTML pages inside your HTML page as the browser are already equipped to handle them.

In AJAX, developer actually need to write code that can parse the XML, extract data and show result.

Oracle Maximum Availability Architecture

I am excited to be going for a 3 day workshop by Oracle on the following week. At present I only aware of Oracle Maximum Availabilty Architecture comprising of the 4 parts:-

• Real Application Cluster(RAC)
• This is to protect for any DB server failure
• Automatic Storage Management(ASM)
• This is to protect from DB Storage failure
• Flashback
• This is to perform a quick undo on user or admin errors.
• Data Guard
• This is for the automated failover of DB system to standby DB site.

I do wonder if the Change Assurance Feature part of this architecture. I hope to learn more and how the MAA is integrated, deployed and what is the best practices.

Private PAAS by Oracle

In recent week, I had the opportunity to attend all three vendors presentation on the topic of  cloud computing. Today, I shall blog about one of them, the Oracle cloud offerings, or they termed it as Oracle Private PaaS(Platform as a Service). I have been using oracle products for many years now, and was very interested in the presentation given by the Oracle reps.

Oracle Private 'PaaS' aimed to achieve these 5 parts.

1) Shared Infrastructure with dynamic scaling
To create an infrastructure which can dynamically scale,  Oracle re-utilized the Grid technology. On the application front, there is an Application grid, where the tools like Weblogic Server, JRockit, Coherence, Tuxedo resides.On the data management, Oracle has the Database Grid & Storage Grid. Concept of In-Memory Database Cache(IMDB), Real Application Cluster(RAC) and Automatic Storage Management are also in built in the private PaaS offerings. The OEM also allow to do the administering.

2) Support component sharing
To support component sharing, Oracle have the SOA & BPM together with the Oracle workflows and Identity Management(IM).

3) Support fast deployment
Pre-configured images, Oracle VM and new component like the Weblogic Server Virtual Edition(VE), can be assembled using the Oracle Assembly Builder by making them into Assembled catalog and then the can be deployed as a platform.

4) Self-serviced
Oracle EM can help PaaS to be self-serviced by scaling up and down.

5) Management and Automation
Lastly management and automation can be done via OEM. IM allow identity management of users.

Oracle believe, most organization providing application services really need a private PaaS solutions instead of IaaS. The focus should be in facilitating the build of apps, making it easier and flexible and can be deploy fast.
Oracle propose this by adopting their Private PaaS solutions.

The cloudburst of IBM

Was fortunate to attend another interesting cloud presentation by a consultant from UK (Mr Lewis Troke) on the cloud offerings by IBM. He was a captivating presenter, whom was very generous to share hs knowledge on cloud computing.At the end, the message that he wanted to put across was that Cloud computing is more business than IT. Technology and IT are still important, but to make cloud computing work, the business must champion it

For this post, I will like to narrow down on "cloudburst." the cloud enabler appliance from IBM.

Cloudburst is actually an IBM hardware appliance that integrates service management software with servers, storage and services for use in the private cloud. It is used to build, configure, deploy and manage the WebSphere private cloud environments. It is also used for building patterns and configuration, which are the Virtual images and script packages so that the deployment can be easier and automated, via the portal.

The virtual images are packaged in the WebSphere Application Server Hypervisor Edition(WAS-HE) under the Catalog. It contains WebSphere components, like the IBM HTTP server, and a Linux operating system. The (WAS-HE) is packaged according to the Open Virtualization Format (OVF) standard.

The steps involved in setting up a private cloud are firstly by integrating IBM cloudburst in the datacenter and network. Then assigning the local storage. Users account and security profiles are also created. The pattern creator can then create some pattern templates. After patterns are created and agreed upon, it will be saved and access list to it will be established. Likely the cloud administrator will then in future deploy this pattern as or when required.

To define the cloud, the administrator normally begin with hypervisors. The details of the hypervisor include the installed URL, authentication information, virtual machine instances and IP groups (DNS servers, gateways, and  range of IP addresses available for the cloud)

The cloudburst can also tracks and reports cloud usage for charge-back based on one's consumption. Other settings include security, backup/restore actions, firmware upgrades, logging, trace auditing and power management.

At the time of this post the cloudburst is at version 1.2.

Oracle VM

I had the opportunity earlier to understand something about OracleVM(OVM) by the Oracle representative that came to visit my company earlier today. Besides the Oracle Cloud-G Architecture that they presented, I am more curious about what is OVM


OVM or oracle virtual machines is based on open-sourced Xen Hypervisor technology. Oracle VM previously only supported on  linux and window platform or they sometime called the, linux and window guests. However, they have recently release Oracle VM server for SPARC or previously in Sun called LDOM.

Each OVM has its own virtual CPUs, network interfaces, storage and operating system. With Oracle VM, users have an easy-to-use browser-based tool for creating, cloning, sharing, configuring, booting and migrating VMs. Using Oracle EM, we can also administered the OVMs.

Oracle has always encourage better use of hardware. So it is not surprise that OVM is built upon predecessor like the Oracle Grid, ASM and RAC. OVM also serve as a building block in Oracle Clouds's offerings.

Installing OVM should be relatively straightforward like installing linux. Give it  a try and it's free.

Links to OVM specs:
http://www.oracle.com/us/technologies/virtualization/024974.htm

Below are some useful OVM commands

Logs path = /var/log/xen/xend 

List which virtual machines are running right now. 
# xm list

Start the vm.  
# xm create vm.cfg

Kill the power to vm: vm01 - hard shutdown 
# xm destroy vm01

Monitors a host and its domains in real time 

# xm top 
 
Send the vm: vm01 the shutdown command, like doing a shutdown inside the vm
 # xm shutdown vm01
 
Current status of the guest operating system is written to disk and removed from system memory
 # xm suspend vm01
 
restore vm01 (any time including after a host system reboot) 
# xm resume vm01

Split / Join files

Sometime the files are too big to put on FAT32 thumbdrives. Use the split and cat commands here:

Split the file system.img into 3Gig chunks 
# split --bytes=3000m system.img

Join up the files system.img.part1, system.img.part2, etc... into the file system.img 
# cat system.img.part* > system.img


Here we created a new device, /dev/xvdb. Now we start up the vm and run fdisk
# fdisk /dev/xvdb

Create a physical volume from this device 
# pvcreate /dev/xvdb1

Create the logical volume group from the physical device 
# vgcreate lvm-oracledb /dev/xvdb1

Create the logical volume using 100% of the free space in the logical group lvm-oracledb 
# lvcreate -l 100%FREE lvm-oracledb -n lvm0

Format the file system 
# mkfs -t ext3 /dev/lvm-oracledb/lvm0

Create a mount point 
# mkdir /oracledb

Add the following to "/etc/fstab" to automatically mount the volume when the computer starts up/dev/lvm-oracledb/lvm0 /oracledb ext3 defaults 1 2
Mount the volume 
# mount -a

ADDITIONAL LINK : 

http://www.techotopia.com/index.php/Managing_Xen_using_the_xm_Command-line_Tool

Remove a Disk (LVM)

Remove /etc/fstab entry[root@vl-ucm1 ~]# vi /etc/fstab

Unmount the device, get device name with: lvdisplay
# umount /dev/lvm-stellent/lvm0

OR
[root@vl-ucm1 /]# ls /u01
downloads lost+found oracle
[root@vl-ucm1 /]# umount /u01
[root@vl-ucm1 /]# ls /u01

If you get: [root@vl-ucm1 /]# umount /dev/lvm-stellent/lvm0
umount: /u01: device is busy

Find out what is using the device with: [root@vl-ucm1 /]# fuser -m /dev/lvm-stellent/lvm0
/dev/lvm-stellent/lvm0: 3557m 3620m 3621m 3622m 3623m 3624m 3625m 3626m 3627m
[root@vl-ucm1 /]# ps auxw| grep 3557
root 3557 0.0 0.5 10856 3104 ? Ss 10:55 0:00 /usr/sbin/httpd
[root@vl-ucm1 /]# service httpd stop
Stopping httpd: [ OK ]
[root@vl-ucm1 /]# umount /dev/lvm-stellent/lvm0
[root@vl-ucm1 /]# df -h
And you should no longer see the disk with 'df'

Remove any entries from /etc/httpd/conf/httpd.conf that refer to this disk.

use lvremove to remove the logical volume[root@vl-ucm1 /]# lvremove /dev/lvm-stellent/lvm0
Do you really want to remove active logical volume "lvm0"? [y/n]: y
Logical volume "lvm0" successfully removed

use lgdisplay and lgremove to remove logical group[root@vl-ucm1 /]# vgremove lvm-stellent
Volume group "lvm-stellent" successfully removed

use pvdisplay and pvremove to remove physical volume[root@vl-ucm1 /]# pvremove /dev/hdb1
Labels on physical volume "/dev/hdb1" successfully wiped

remove partition with fdisk /dev/hdb, 'd', enter, 'w' = write partition and exit
# fdisk /dev/hdb

Remove the disk entry from the vm.cfg file, and restart the vm.

To shrink a LVM

Just create a new disk, copy over contents and delete old volume.
.

Looking at web world crystal ball

Where are we heading, so many technologies, so many resources.. certainly we have so many choices. Or are we getting lost in the world of web. Let make some assumption on the way forward for web world.

[1] The world of web will continue to evolve

[2] Dominated by technology-based companies or Information commodity markets.

[3] Machines will keep getting cheaper, but demand and expectation keep getting higher

[4] Everything getting connected

[5] Users will continue to be human

We will see our web and world getting more connected and maybe as important as electricity.

10 Habits of Successful Web 2.0 Sites

[This writeup was part of my technical sharing presentation that I was preparing for fellow infrastructure sysadmins]

#1 Contribute First
- Aim to provide a site that can contribute to user, free of charge.
- Share the data as much possible.

 #2 User-centered Design
-  Simple and ease of use is still important feature of anything, including Websites.
- Allowing customizations and personalisations.
- Collect users feedback(social understanding) and make the enhancements.
  
#3 The Web as Platform
- The web is the BIG platform. Use it!
- Accessibilty from any methods possible.
- Creating something that could be a potential platform or foundation for others, will definitely be a hit! (e.g Facebook)

#4 Crowd-sourcing
- Allowing user to participate and contribute. Even a small contribution, with millions of such contribution will eventually lead to a relevant website
- Focus on attracting people to contribute and share. The more people use it the better it gets.

#5 Rich User Experience
- Faster,lighter,less clustered, service oriented.
- Using AJAX, CSS2.0, XHTML, AIR

#6 Dynamic Content
- Plan and aim for the Web 2.0 services to be proactively growing and maturing. 
- With crowd-sourcing, web 2.0 can access and facilitate the dynamic content much easier than before.

#7 Mashup & Collaboration
- The power of networking and joining forces can be good. Combining two great web 2.0 applications will create a better Mashup applications.

#8 Power Decentralisation
- Web 2.0 is about people. The power is with the people. Allowing people to contribute, control and trust will be the way forward to grow the Web Ecosystem.

#9 Cloud computing - Software as a Service (SaaS)
- Elasticity and scalable.
- Platform independent

#10 It is all about the users.
- Users are the key component of any web 2.0 sites. Without them, the site fail.
- Make users part of the architectural framework.

Semantic web - Web 3.0

The semantic web is a vision of information that is understood by computers(machine), so that they can perform more of the tedious work involved in finding, combining, and acting upon information on the web.

At present, only humans are capable of using the Web to carry out tedious tasks. However, a computer cannot accomplish the same tasks without human direction because web pages are designed to be read by human, not machines.

Tim Burner Lee consider the semantic web as the next web 3.0.

W3C and many others are rushing to come out with design principals and standards.

Some of these include Resource Description Framework (RDF), a variety of data interchange formats (e.g. RDF/XML, N3, Turtle, N-Triples), and notations such as RDF Schema (RDFS) and the Web Ontology Language (OWL), all of which are intended to provide a formal description of concepts, terms, and relationships within a given knowledge domain.

Tools used in the making of facebook

Facebook is the #1 social-networking site in the world now. Did you wonder what are the tools that made facebook so successful. Beside people and data, which are the key ingredient that help to make facebook so yummy, there lies all those utensils that make it happen. Today, I shall focus on some tools that are use  for Facebook. Most of the description here are extracted from facebook developer site or developer wiki sites.


Scribe - An open source server for aggregating log data streamed in real time from a large number of servers. It is designed to be scalable, extensible without client-side modification, and robust to failure of the network or any specific machine.

Scribe server running on every node in the system will aggregate messages and send them to a central scribe server. If the central scribe server isn’t available the local scribe server writes the messages to a file on local disk, and sends them when the central server recovers. The central server can write the messages to the files that are their final destination, typically on an nfs filer or a distributed filesystem, or it could also send the messages to another layer of scribe servers.

Clients log entries consists of two strings- a category and a message.

Facebook and Twitter is currently using scribe.

Source code at http://github.com/facebook/scribe



Thrift s a software framework(libraries) for scalable cross-language services development. It combines a powerful software stack with a code generation engine to build services that work efficiently and seamlessly between C++, Java, Python, PHP, and Ruby. Thrift allows you to define data types and service interfaces in a simple definition file. Taking that file as input, the compiler generates code to be used to easily build RPC clients and servers that communicate seamlessly across programming languages.

http://wiki.apache.org/thrift/FrontPage


OpenLink Data Spaces (ODS) is a Distributed Collaborative Web Application Platform, Social Network, and Content Management System for creating presence in the semantic web via Data Spaces derived from Weblogs, Wikis, Feed Aggregators, Photo Galleries, Shared Bookmarks, Discussion Forums and more.


Data Spaces are a new database-management technology frontier that deals with the virtualization of heterogeneous data and data sources via a plethora of data-access protocols.

As Unified Data Stores, Data Spaces also provide solid foundation for the creation, processing and dissemination of knowledge, making them a natural foundation platform for the emerging Data-Web (Semantic Web, Layer 1).
 
Data Spaces also provide a cost-effective route for generating Semantic Web Presence from Web 2.0 and traditional Web data-sources, by delivering an atomic data container for RDF Instance Data derived from data hosted in Blogs, Wikis, Shared Bookmark Services, Discussion Forums, Web File Servers, Photo Galleries, etc.
More info please refer to:-

http://virtuoso.openlinksw.com/dataspace/dav/wiki/Main/Ods

Facebook Markup Language is a markup language that is used to customize the "look and feel" of applications that developers create.

It is a variant  of HTML with some elements removed. This allows Facebook Application developers to customise the "look and feel" of their applications, and how to encode content so that Facebook's servers can read and publish it.

FBML set by any application is cached by Facebook until a subsequent API call replaces it. Facebook also offers specialised Facebook Javascript (FBJS) and library.

FBML Tags

• Social data tags
• Sanitization tags
• Design tags
• Component tags
• Control tags

Facebook Platform wiki site:
http://en.wikipedia.org/wiki/Facebook_Platform


FB Development Tools
Facebook Application Programming Interface (API)
Facebook Query Language (FQL)
Facebook Markup Language (FBML)
Facebook JavaScript (FBJS)
Facebook Connect

Local Cloud, virtualization and more.

Even as cloud computing has been around for quite awhile, most organisation  will likely not to jump straightaway to cloud. Most will approach it first using hybrid method. Moving the least sensitivity and lesser risk apps to cloud while majority applications are still retain in the traditional computing system. It will then evolved in phases. Some organisation will then start to offer to user the computing resources as utility based product. Let's take a look at virtualization as it is the main driver of cloud computing.

Virtualization witin the organisation can be segmented to three types:-

• Server Virtualization
• Using VMware, Hyper-V, Zoning
• Application Virtualization
• Software technology that separate from underlying OS
• Presentation Virtualization
• example is "Thin Client"

Type of server virtualizations

• Microsoft Hyper-V based on window 2008 server
• VMware based on linux & window server
• VMware ESX Server 3i, 3.5
• VirtualCentre 2.5

 Type of thin client

• Sun's Sun Ray ( Sun Virtual desktop Infrastructure (VDI)
• HP VDI  & 6720t Mobile thin client(including Neoware)
• Dell ( e.g OptiPlex & Flexible computing solution)

Why Virtualised?

• Cost
• Reduces administration
• Fast Deployment
• Reduced Infrastructure Cost 

Not suited for cloud :

• Security sensitive data & app
• Graphic Intensive app
• Business Intelligence and data warehousing

Best Practices 

• Find the right vendor
• Read the fine print
• Measure the performance
• Spread service across more than one reliable cloud vendor
• Data should be easily view & port over by owner
• Data security and data flow
• Phase by phase approach

#################################################
Based on Gartner's prediction on Cloud Computing Market

Phase I : 2007 -2011 (Pioneer & Trailblazer)

• Cloud Market development phase, and SEAP(service enabled application platform)

Phase II : 2010 -2013 (Could Market consolidation)

• By 2012, SEAP market will be overcrowded with broad range of cloud solutions

Phase III : 2012 - 2015 & beyond (mainstream critical mass & commoditization)

• Cloud market matures and small dominating cloud player providing de facto standards.

References:
Cloud Computing- A Practical Approach, by Anthony T.Velte, Toby J Velte & Robert Elsenpeter (McgrawHill)

Standard used in cloud computing

Communication

• HTTP - http1.1
• XMPP(Extensible Mesaagingand Presence Protocol) also knownas "jabber")

Security

• SSL
• OpenID
• PCI DSS (Payment Card Industry Data Security Standard)

Client

• HTML
• Dynamic HTML (DHTML)
• Document Object Model (DOM)
• Javascript

Infrastructure

•  Virtualisation
• Open Hypervisor Standards
• Community Source (e.g in VMware ESX server)
• Open Virtualization Format (OVF)

Web Services

• REST(Representational state transfer)
• SOAP (Simple Object Access Protocol)
• JSON(Javascript Object Notation)
• XML

Idea of Web 2.0

Idea:

• The Web as a platform ( Web is the interconnection of applications and devices)
• Collective Intelligence is harnessed
• Valuable Data commodity
• Software that cross boundaries
• It gets better the more users used it
• Continuous growth and improvement
• Social rich-user experience

 Offshoots from web 2.0

• Enterprise 2.0
• Government 2.0
• Law 2.0
• Library 2.0
• Media 2.0
• Advertising 2.0
• Music 2.0
• Identity 2.0
• Democracy 2.0

Cloud Computing and the major players

Google

• Google App Engine -empower developer to build web-apps on cloud infrastructure
• Google Web Toolkit - a PaaS, tools based on Java with AJAX support for development & debugging

Amazon

• EC2 - Elastic Compute Cloud is a web service that offer elasticity capcity in cloud.
• SimpleDB
• S3 - Simple Storage Service is Amazon's cloud storage solutions
• CloudFront - webservice for content delivery on cloud.
• SQS- Amazon'z Simple Queue Service is a cloud messaging system
• EBS - Elastic Block store is Amazon's persistent storage

Microsoft

• Azure Service Platform ( cloud services platform hosted by microsoft)
• Exchange Online
• Sharepoint services
• Microsoft Dynamic CRM

Saleforce.com

• Force.com - on demand PaaS 
• Visualforce - tool fordesigning  business apps
• AppExchange - directories of apps for salesforce.com by 3rd party.
• Salesforce.com CRM

EMC

• VMware 
• Symmetric V-max system - Management system to manger virtual datacenters
• Storage leader

NetApp

• Cloud Storage & data management solutions
• Unified storage architecture (Joint with Cisco)

Cisco

• Unified Computing System

IBM

• IBM Global Business Services
• X-Force

SAP-IBM

• Partnership with IBM on project RESERVOIR(REsource and SERvices Virtualisation withOut barrIeRs)

Yahoo

• Yahoo! Research & CRL(Computational research laboratories)

HP-Intel-Yahoo!

• creating a global multidatacenter, opensource test bed for cloud computing research.
• partnership with Singapore's IDA, Universitiy of Illinois & Karlsruhe Inst of Technology(Germany)

FB Architecture - Brief

One of the most, if not the most often used social networking site, facebook. Today, I will try to summarize the Facebook(FB) architecture of what I have understood from Aditya(Dir. Engineering, Facebook) talk in Dec 08. (Note: this is my intepretation of the architecture look from the talk.)

Basically FB utilized the LAMP framework, but the modified version, LAMP stands for Linux, Apache, Mysql and PHP/Perl. The wise people at FB has modified and enhance most of the LAMP components, especially Mysql & PHP while including additional services to create FB. Web 2..0 tools like the Scribe, Thrift & ODS  were employed in developing the FB. A giant hash memory table or Memcache also helped to create a fast responsive FB. Services are like, newsfeed,Adserver, Search, network selector,CSS Parser, ShareScrapper, mobile  and blogfeed make up the complete FB architecture.

A notable mentioned in FB design is 'memcache'. Why memcache?. Its is faster and of high performance, alleviate the load on databases, cache serialized PHP data structures and multi-get to retrieve data. FB has modified memcache to run over UDP reducing the overheads and the persistent connection of TCP.

One key note that data can be slightly delay in facebook and need not be accurately realtime, although FB will try to be consistent. Data is also sorted in recentcy layout for optimizations. Data of the form key-value pair is evenly distributed across multiple DB instances and used LB. This allow FB to scale fast and do not require replications. The query in FB application is often a simple query type and they do not allow joins, and need not too.

One thing to always remember is that FB is not just a site about social-usages, but it also allow others to contribute and grow its applications. Developer can write FB apps, and using FBML extensions to easily interface and publish program. Of course the FB application's one developed has to be hosted somewhere other than the FB site. This flexibility allow fast adoption and wide use of FB.

How oracle start its database.

Let get into the backend scene that happen when we start an Oracle database. First assuming the system have already installed and create an Oracle Database.

Pre-requisite

• ORACLE_HOME is defined ( normally is the Oracle DB install path)
• ORACLE_SID is known (The Instance DB that we want to connect)

Processes

1. Setting up the Oracle environment for the user.(Oracle DBA User, ORACLE_HOME & ORACLE_SID)
2. Using "oracle" dba account issue >> startup [db]
3. Oracle will lookup the spfile [$ORACLE_HOME/spfile$ORACLE_SID.ora]<-- binary file
4. With the spfile, oracle will then create SGA (System Global Area) in the memory.
5. All the oracle processes will also be started for the instance.(DBWR,LGWR,SMON,PMON.etc)
6. The SGA will be assigned spaces for 
• Fixed Buffer
• Variable Buffer
• DB Buffer
• Log Buffer
7. Next oracle will look for the control files that is mentioned in the spfile. Control files is crucial as it allow for
• location of 'df files like datafiles','redofiles','tempfiles' & 'logfiles'
• check global db file consistencies
• Check if need to do any rollback
• At the point the datafiles will be known.
8. Database can then be mounted
9. Next Database will then be open
10. If the connections are from other systems or client users, we need to start the listener (LSNR) >> lsnrctl start

Basically this  are the normal steps an oracle database goes through during a database start.

Oracle Database Physical & Logical storage architecture

Oracle storage design of database architecture is interesting and good. The concepts of having Physical & Logical components are really helpful. For example, DBAs can maintain the physicals structures while the logical instance are still running.

Clearly isolating this two tiers are helpful to DBA & developers. Developer do not need to remember which data file that the database is assigned. Administrator can scale the Tablespaces as and when needed to expand or reduce by adding or reducing the data files.

PHYSICAL

Datafiles

• Take note : A datafile can only be assigned to a tablespace, but a tablespace can have many datafiles

• When ever user request data, oracle retrieve that data from these datafiles with the help of processes in the SGA (System Global Area). Manipulated data is written back to these datafiles, so that changed data is available to all users.

Beside the physical data files, there also other physical files like the redo log files, and control files.

Redo Log files

• The redo log files store the change data entries generated by DML. 
• It will be used during database recovery processes. 
• Ofter  redo log files  are  also archived, and it is copied offline for recovery purpose. (archived redo logs)

Control Files

• Control files is important file to start the DB instances.It contatin the initiatisation info of the DB(init.ora)
• It record information about physical structure of database, such as datafiles size and location, redo log files location.

LOGICAL

Tablespace

• Tablespace is a logical structure which holds other logical structure of database. 
• This is further broken into segments. segment is further broken into extents, and extent intodata block. 
• Tablespaces have two main type: Data & Index TS and Undo TS.

Segments

• Table in the database will store into a  Data Segment.
• Index in the database will  be store in Index Segment. 
• Temporary Segment
• Rollback Segment.

Extents

• A segment is further broken into extents. 
• An extent consists of one or more data block. 
• When the database object is enlarged, an extent will be allocated. 
• An extent cannot be named.

Data Block

• Smallest unit of storage in the database. 
• The data block size is a specific number of bytes within tablespace and it has the same number of bytes. 
• It consist of multiple operating system blocks.

Architecture concept for web 2.0

Architecture has made design and building IT into a more consistent and manageable approach. I have recently study some architectural patterns and perhap it is a good idea for my record purpose to blog this web 2.0 pattern in for future reference.

Basically there 5 main component groups for web 2.0.

Design &
Devolopment   --> Client/App Tier
                                      |
        |                      Connectivity Tier
        |                              |
        |                      Service Tier
        |                             |
     [ Resource Tier               ]


Client/App Tier
This tier allow users to interface the services. It is where users interact with the web 2.0.
Component such VM,portals,media renditions, security controls maybe present in this tier.

Connectivity Tier
This tier support the standard connection protocols that support all the connectivity matter in the architecture. Most often the technology is define by community experts like W3C, OASIS, etc.(e.g like XML/HTTP)

Service Tier
This is where the resources is package and group as a service. This is also where BusinessRule(BR) and workflow is added.( e.g PHP, ASP, Rails, SOAP)

Resource Tier
This tier support the repository of data and support processing that is needed to create Rich Internet Application.(ERP,CRM,DB,MQ, LDAP,LegacySystem)

Design & Development
This architecture tier support the development of all the other 4 architectural components by providing standardization and control in the Integrated Development Environment(IDE) for example.

Web 2.0 quotes

" Web 1.0 was about connecting computers and making technology more efficient for computers. Web 2.0 is about connecting people and making technology efficient for people."

Dan Zanbonini

" The Internet is a platform spanning all devices " 

Tim O'Reilly, 2005

"Trust I seek and I find in you

Every day for us, something new

Open mind for a different view

and nothing else matters."

(Nothing Else Matters" by Metallica

"If you build it.. they will come"

Field of Dreams

Institution takes on Cloud.

I had the opportunity to catch a short seminar by Tan Chee Chiang, Assoc. Director, High Performance Computing, Computer Centre, NUS. Briefly here what I summarised from the interview:-

From institution point of view cloud computing:-

• Allow for high density servers requiring special power and cooling
• Short term peak during term end, and project peak period
• Agility to function like a rubber-band in providing services

Challenges:-

• Stable connectivity between consumer and the cloud service provider.
• Data security in the cloud.

Some common Web 2.0 Patterns

The following are some common architecture patterns.

• Service Oriented Architecture(SOA)
• Participation Collaborations
• Software as a Service(SaaS)
• AJAX( or Asynchronous component update)
• Rich Internet Application(RIA)
• Synchronized web
• Mashup
• Folksonomy
• Tag Gardening
• Semantic Web Grounding
• XML(Structured Information)
• Persistent Right Management

Why Web 2.0 is what it is..

" Web 2.0 is the network as a platform, spanning all connected devices; Web 2.0 applications are those that make the most of the intrinsic advantage of that platform:-

• delivering software as a continually updated service that gets better the more people use it,
• consuming and remixing data from multiple sources, including individual users while providing their own data and services in a form that allows remixing by other,
• creating netwrok effects through an "architecture of participation" and
• going beyond the page metaphor of Web 1.0 to deliver rich user experiences.", Tim O'Reily

With rapid adoption of broadband internet and nation rushing to implement high speed next generation networks, content providers are able to leverage on Rich Internet Applications and provide Web 2.0 sites. Mobile devices like iphone, androids, and many more devices to come has in a lot of ways accelerate the adoption of Web 2.0.

The concept of user participation's leads to viral marketing, where advertisement and marketing can be promoted faster and wider than ever before. Content can be even created by users to drive the product.

But Web 2.0 is more than just about internet technology revolutions. It is a change that has affect human to human interactions, business or personals. And it has become an integral part of peoples life, at least those staying in areas with internet connectivity. Part of the success recipe in web 2.0 is the inclusion of users as a critical ingredient which engage and provides key functionality and content.

Not just the Economics driving web 2.0, Politics are well in this space to utilise the participatory, netizen based platform that has become the truth. Mainstream media like the press/tv news that used to dominate starts to loose foothold in spreading information. With Web 2.0, an average person can participate, contribute and dominate the web 2.0 world. As evidence in the US president election in 2008, the wide use of Facebook, and other social-networking platform has aided in canvassing and garnering support. This has certainly change the way the world function today and web 2.0 has certainly nudge the balance of power toward the average person.

The pace of innovation and introduction of technologies is ever increasing. We are in these exciting phase of the world era, so just accept it and called it Web 2.0 world.

Augmented Reality

These words is becoming more norms as mobile devices, game consoles and even websites, start to use and drive the Augmented Reality technology faster and faster. Games console like Wii, Apple iPhone a few common augmented reality tools has been experimenting and creating applications to enhance sound, sights, smells and effects by creating a new world where reality meets computer generated layers.

I guess all the regular BPL fans will attest see-ing the famous Barclay's logo at the center of the football field just before the kickoff. In reality, the logo was not painted on the grass, it is where the advance of the computer augmented reality concept creating a layer ads over the football field. The offside line now can easily be replayed and marked with a shade using this video augmented techniques. The applications are wide. For the football case, advertisements revenue can be grown and expanded using this technology. The sideline boards can be reconfigure with advertisement pertaining to the locality. TV stations can then subsidise using part of the ads revenue on the match itself and thus giving a better cost charging to consumers. In nintendo Wii, we saw how various Wii consoles can emulate devices creating the impression of actually acting the part like playing Tennis or bowling. Some website like Wikitude is starting to use location info along with camera phone to cleverly interpret places of interest.

Basically Augmented Reality is getting to be the norm. In Japan some movie even have sensory responders to give certain scent during the movie to create a more realistic appreciation of the movie. Universal Studios 4D shows will sprayed out water if there screen showing running down the speedy boat, and make blow heat to signify if you adventuring up the volcanoes.

While we let augmented reality catch up with the reality, I just had to mentioned the scene from the movie Minority Report where augmented reality becomes more real than sometime reality itself. In that future, likely the line between realities are no longer there.

IT Service Building Block

Service building block comprises of software components. It can be group into 3 areas.

Presentation <---  Connectors  <----  Services

• Presentations
• HTML pages
• Java servlets
• JSP
• AJAX
• Connectors
• JMS( Java Messaging Services)
• JNDI ( Java Naming & Directory Interface)
• JDBC ( Java Database Connector)
• JNI ( Java Native Interface)
• RMI (Remote Method Invocation)
• JAXP (Java API for XML Parsing)
• JCA ( Java Connector Architecture)
• Services
• Order Entry
• CRM (Customer Resource Management)
• ERP (Enterprise Resource Planning)
• OLTP
• OLAP
• DSS(Design Support System)
• HRIS(Human Resource Info Sys)
• Remote Services
• Provisioning

Infrastructure Tiers

• Presentation Tier
• Web Server
• Portal Server
• Cache Server
• Print Server
• WAP Server
• Business/Application Tier
• Application Server(Weblogic, WAS)
• Mail Server(SMTP,POP)
• Fax Server(Hylafax)
• SMS Server
• Integration Tier
• Messaging Server(MQ)
• Directory Server(LDAP, Exchange)
• Resource Tier
• Database Server
• FTP Server
• NFS Server
• Directory Server
• Report Server 
• Replication Server
• Systemic Tier
• Load Balancing Server
• Certificate Server(SSL)
• Security Server(Syslog-ng, IDS-Intrusion Detection Server)
• Monitoring Server(Nagios, CA-Health, Introscope)
• Time Server(NTP)
• DNS Server
• Backup Server

AIR

AIR has been around for few years now and there more than 100million AIRapps out there. So what exactly is AIR, since most of the modern website often required patrons to accept the AIRapps.

From wiki definition on 2 March 2010,(Extracted from WIKI)
AIR or Adobe Integrated Runtime is a cross-platform runtime environment developed by Adobe Systems for building rich Internet applications using Adobe Flash, Adobe Flex, HTML, or Ajax, that can be deployed as a desktop application.


AIR can be operated offline, and when Internet connection is made avalaible again the function will get activated or updated or you can also upload data.

AIR helps web developer as a tool to build rich internet apps.

RSS

What is RSS.
RSS is a Web content syndication format, which in long stands for  Really Simple Syndication.

Browsers can read directly RSS files, or you can also used RSS reader or aggregator.

RSS is a format to share information in an XML file. Currently RSS is at ver 2.0 controlled by W3C.

Example as follows

1) An xml based file(rss feed) to be uploaded to the website

2) Use the following link "http://www.feedvalidator.org/" to validate the rss feeds.

3) Add the link to deploy the RSS in the page.

Folksonomy

Folksonomy:

- is done by the person using the information.
- is about free tagging of information and objects.
- is about tags used for retrival. 
- tagging is done in a shared and open social environment.


Three ELEMENTS in folksonomy
1) Person tagging
2) Object being tagged
3) Name of the tag


The Values
- the tagging is derived from people using their own vocabulary and adding explicit meaning,
- Allowing people to participate in the tagging in itself create value to the Internet world  and by which making folksonomy an important tool in Web 2.0

Web 2.0 => Enterprise 2.0

Web 2.0 media tools like Twitter and Facebook and applications such as wikis, blogs and forums have allowed collaboration and knowledge management in which helps to transform companies into an Enterprise 2.0 organisation.

With wikis and blogs, employees can participate/exchange their experiences and expertise in the way of sharing knowledge. This also benefits the organization in term of knowledge management and a participative employees usually translate into a motivated and progressive workforce. It also allow knowledge to be available across multiple location and support the globalization workforce of today.

Of course the concern & challenges of enterpise 2.0 will then be of security, governance and ethics.

Mashup

Mashup is a webpage or an apps that combines and uses data & functionality from two or more sources. There is 3 types of Mashup.


1) Consumer Mashup
Combines different data types, example GoogleMap and Wikipedia APIs

2) Data Mashup
Combines similar data types but different data sources to create a new distinct webservices.

3) Enterprise Mashup
Combines external data information with the enterprise own resources, applications and data creating a rich collaborative presentation view. Example Agile Development.


Architectural aspects:
- Web-based : mashup done on the web browsers
- Server-based : mashup done on the server and then transmitted as a reply to the requesting browser.

Cloud Model Comparison

According to the layers from top to bottom, at top most we have Software as a Service, follow by Platform as a Service and bottom most Infrastructure as a Service.

First we discuss SaaS. SaaS is essentially providing software traditionally need to be purchase by licenses into a pay as you used model.

Software as a Service(SaaS)-  Cloud Applications

Complexity : Low
Flexibility : Minimal Control, determine by apps provider, minimal customization
Consumers: End Users
Type of services: Webmails, MessageLabs, Virus Scanners, online office suite-zoho, CloudFront etc
Pricing: Price based on per used basis.
Benefits: 

• Save Time & Cost, 
• Budget focus of competitive advantage instead of infrasturture,
• Utility model of pay as you used, without worrying of licenses or version upgrades
• Faster to start use, no need to install.
• Flexible & Scalable and 
• Multi-tenant efficiency.

Examples: GoogleApps, Salesforce.com, Cybershift, Zoho, TurboTax Online, Microsoft Online Services, MailTrust.



PaaS provide a platform for business to quickly churn out applications without worrying about setting up servers and development tool licenses. Sometime SaaS can also ride on top of PaaS

Platform as a Service(SaaS)- Cloud Platforms

Complexity : Middle

Flexibility : Medium Control

Consumers: Developers, Programmers,

Type of services: development system, debugging and testings, deployment

Pricing: Price based on per used basis.

Benefits:

• Multi-location development sites working together possible 
• Multi interaction development
• Cost reductions derived from using built-in infrastructure services
• Cost reductions derived from using highlevel programming abstractions
• Opensource and standardization possible
• Multi-tenant efficiency.

Examples: Amazon Web Services, GoogleAppEngine, Force.com, WindowsAzure, engineyard(based on rails on cloud), WolfFramework

(below is the comparison chart extracted courtesy of zdnet.com article comparing "Amazon's and Google's Platform-as-a-Service (PaaS) Offerings")

IaaS abstract the computer infrastructure of a system. It can sometime be also refers as Everything as a Service model as it can also include PaaS and SaaS in it.

Infrastructure as a Service(SaaS)-  Cloud Infrastructure

Complexity :High 

Flexibility : Full Control

Consumers: Systems Administrators

Type of services: development system, debugging and testings, deployment

Pricing: Price by utilisation, size, performance.

Benefits: 

• Resources delivered as a service
  
• Dynamic scaling of infrastructure(Elasticity) 
• Lower CapEx as infrastructure is used as service
• Enterprise graded infastructure can be used by mid or small size company to benefit due to aggregated sharing
• Multi-tenant efficiency.

Examples: Amazon EC2, Nebula NASA, Flexiscale

---------------------------------------------------------------------------------------------------------------

Annex A

comparison chart extracted courtesy of zdnet.com article comparing "Amazon's and Google's Platform-as-a-Service (PaaS) Offerings"

 

Web 2.0 - Traits & Challenges

Traits:-

• Rich User Interface
• Users are the Points
• Data Super Transforming-Imagining-Ability
• Continous Improvement

Challenges:-

• User's Interface
• User Centrics
• RE: Imagining your UI
• Free your Data

Web 2.0 apps => "Applications that get better the more that people use them."


Web 2.0 Technologies:-

• AJAX
• AIR( prev called apollo)
• Web Standards
• User Centred Design
• RSS
• APIs
• Blogs,Twitter,Facebook
• Mashups
• Social Networking
• Tagging(Geotagging)
• Folksonomies
• Cloud

What is REST?

Roy Fielding describes Representation State Transfer(or REST) as an important architectural feature that define the World Wide Web(WWW) today. 

"REST emphasizes scalability of component interactions, generality of interfaces, independent deployment of components, and intermediary components to reduce interaction latency, enforce security, and encapsulate legacy systems."

Basic Flow of simple REST implementation.

i) Online publisher made available content(e.g in XML)

ii) Subscriber, using the URL, read the XML via browser, 

iii) data gets interpreted and reformatted and re-packaged to be use.

REST has advocate the requirement for a uniform interface.

REST requires the use of a common, generalized, generic interface. It does not permit application-specific interfaces. 

In the WWW, the REST interface is defined by the HTTP interface, which supports four basic application methods: GET, POST, PUT, and DELETE."

Service-Oriented Architecture- Definition

SOA or service-oriented architecture basically is a collection of services focusing on services. It has been around for many years now. In my opinion, it is a concept and future post I will try to explore the value of SOA.
There are many SOA frameworks/models define and available on the net. As for now I leave the post with this definition found on wiki.

One of the strong member for SOA,OASIS defines SOA as the following:
A paradigm for organizing and utilizing distributed capabilities that may be under the control of different ownership domains. It provides a uniform means to offer, discover, interact with and use capabilities to produce desired effects consistent with measurable preconditions and expectations

WOA - Web Oriented Achitecture

From wiki,

Web Oriented Architecture (WOA) is a style of software architecture that extends service-oriented architecture (SOA) to web based applications, and is sometimes considered to be a light-weight version of SOA. WOA is also aimed at maximizing the browser and server interactions by use of technologies such as REST and POX.

The build systems of arbitrary complexity, much of the similarity ends there. Here are some of the most significant contrasts between the two approaches:

• SOAs tend to have a small and well-defined set of endpoints through which many types of data and data instances can pass. WOAs tend to have a very large and open-ended number of endpoints; one for each individual resource. Not an endpoint for each type of resource, but a URI-identified endpoint for each and every resource instance.
• Traditional SOA builds a messaging layer above HTTP using SOAP and providing unique and sometimes prohibitive constraints to the Web developer, while WOA finds HTTP and related transfer mechanisms to be the ideal layer of abstraction for most applications.
• SOA was designed from the top-down by vendors to be tool friendly, while WOA was emerged form the bottom up from the Web naturally and has the best support in simple procedural code and an XML parser.
• SOA uses WS-Security and other sophisticated standards for security, while WOA tends to just use HTTPS.
• SOA must contend with the vagaries of XML Schemas for service contracts, while WOA largely ignores the issue and lets Web services naturally represent whatever formats are desired.
• Traditional SOA is fairly cumbersome to consume in the browser and in mashups while WOA is extremely easy to consume just about anywhere

Hello Web 2.0

I just like the title to call my first post on Web 2.0. (It reminds me of the countless hello world examples)

I was recently asked to prepare a presentation on a subject of  Web 2.0 to my fellow colleague in a month time. As I was researching on this subject, I can't help but to get lost in this world of Web 2.0. There so many interpretations of what is Web 2.0, and some even say it does not exist or a hype.

Actually no one coined the term web 1.0 prior to web 2.0.
So the story goes like, once upon a time there was just the Internet as it is/was or we call it the arrival of Internet.(Now the IT community refers it as Web 1.0)
Web 2.0 essentially is second revolution or next phase of internet world, which is happening now.

In my opinion web 2.0 should encompase the following:-

• User or community involvements. (allow participation and content creation)
• Real time (or near realtime web interaction)
• Service Focus ( Web apps is build following Service Oriented Architecture)
• Network effect ( which include enabling techniques like AJAX & cloud computing)

I will stop here for now on the brief glance at web 2.0.

Puppet : What, Why & How

What's the intention?

• To help the sysadmin community to build and share tools.

 Why use Puppet?

• Provides a powerful framework to simplify the technical tasks that sysadmins need to perform.
• Sysadmin work is written as Puppet’s custom language code which is shareable and reusable.
• Written as a comprehensive CM(configuration management) tool.

How to implement the architecture.

A simple star topology architecture as follow can be a good point to start
- A versioning server is required to assist in the CM.
- One server as the master Puppet Server.
- Other client server running the puppetd agent.

Introduction to Puppet

Puppet a sysadmin tool look set to be the "in" thing for sysadmin. It helps to handle configuration management and it is opensource.

From reductivelabs definitions:

Puppet is an open-source next-generation server automation tool. It is composed of a declarative language for expressing system configuration, a client and server for distributing it, and a library for realizing the configuration.

Created by Luke Kanies Reductivelabs

Puppet is a system for automating system administration tasks.

Links

• Puppets Download(http://reductivelabs.com/trac/puppet/wiki/DownloadingPuppet)
• Puppets Recipes(http://reductivelabs.com/trac/puppet/wiki/PuppetRecipes)
• Puppets documentation(http://reductivelabs.com/trac/puppet/wiki/DocumentationStart)

Introduction to AJAX

AJAX or Asynchronous Javascript And XML is technology that facilitate the interaction with database or other external data without the need to reload the webpage.

This is made possible by the DOM API called xmlHttpRequest, which allows HTTP/HTTPS requests to be initiated and handled from within Javascript.

Simple Example of AJAX implementations

Reference from (Sample from cloudwisp.net)

Components

• HTML with embedded Javascript file (*.js)
• Database Table(e.g shoutOutTable)
• PHP/ASP file to query the table (e.g PHP-getShoutOut.php)

/////////////////////

//function to place in your .js(javascript) file for the current document(html)

////////////////////

function xmlRequest(){

var xmlHttp=null;

if (window.XMLHttpRequest) {

// code for IE7+, Firefox, Chrome, Opera, Safari

xmlHttp=new XMLHttpRequest();

}

else if (window.ActiveXObject) {

// code for IE6, IE5

xmlHttp=new ActiveXObject("Microsoft.XMLHTTP");

} else {

alert("Your browser does not support XMLHTTP!");

}

return xmlHttp;

}

function getShoutOuts(){
 var xmlObject=new xmlRequest();
 xmlObject.open("GET","getShoutOuts.php",true);
 xmlObject.send(null);
 xmlObject.onreadystatechange=function(){
  if (xmlObject.readyState==4){
   var returnString=xmlObject.responseText.split("|");
   if (returnString[0]=="OK"){
    //query was successful
    var returnedHTML=returnString[1];
    document.getElementById( ishoutContainer).innerHTML=returnedHTML;
   } else if (returnString[0]=="ERROR"){
    //query failed in php and reported an error
    alert(returnString[1]);
   }
  }
 }
 var t=setTimeout(getShoutOuts,10000);
}

/////////////////////

Database Table(shoutOutTable) in mysql

soID(int,auto_increment),userName(varchar 50),shoutOutMsg(varchar 255)


PHP File (getShoutOuts.php)

include "connect.php";
$sql="SELECT * FROM shoutOutTable ORDER BY soID DESC LIMIT 10";
$result=mysql_query($sql);
if (mysql_errno()){
 echo "ERROR|".mysql_error();
} else {
 echo "OK|";
 for ($row=mysql_fetch_assoc($result)){
  echo "".$row['userName'].": ".$row['shoutOutMsg']."";
 }
}

// this will return latest 10 shoutouts

Additional Info

xmlObject.readyState

The "readyState" property keeps track of the current stage of the request by returning an integer:

• 0: uninitialized
• 1: loading
• 2: loaded
• 3: interactive
• 4: complete

Cloud Service Model

 Software as a Service (SaaS). The capability provided to the consumer is to use the provider’s applications running on a cloud infrastructure. The applications are accessible from various client devices through a thin client interface such as a web browser (e.g., web-based email). The consumer does not manage or control the underlying cloud infrastructure including network, servers, operating systems, storage, or even individual application capabilities, with the possible exception of limited user-specific application configuration settings.


Platform as a Service (PaaS). The capability provided to the consumer is to deploy onto the cloud infrastructure consumer-created or acquired applications created using programming languages and tools supported by the provider. The consumer does not manage or control the underlying cloud infrastructure including network, servers, operating systems, or storage, but has control over the deployed applications and possibly application hosting environment configurations.



Infrastructure as a Service (IaaS). The capability provided to the consumer is to provision processing, storage, networks, and other fundamental computing resources where the consumer is able to deploy and run arbitrary software, which can include operating systems and applications. The consumer does not manage or control the underlying cloud infrastructure but has control over operating systems, storage, deployed applications, and possibly limited control of select networking components (e.g., host firewalls).

[ This definition is provided from NIST(National Institute of Standards & Technology -US)]