Something really worth reading if you are into Cloud Computing !! Awesome Blog !!
How Big will the Internet be in 2015?
Answer, pretty big, no let me re-phrase, ginormous!!
In the following infographic, Cisco try to demonstrate just how big the web is going to get in only a few short years. As the content we consume gets bigger, more visual and of higher quality the amount of data passing through servers, down phone lines or over the air will continue to grow exponentially.
Adobe Creative Cloud: Creation On The Cloud
Sometime back, creative software giant Adobe released the long-awaited CS6, which comes with Creative Cloud.
Creative Cloud provides a simple subscription option ($50/month) to access Adobe’s complete Creative Suite (Photoshop, Illustrator, Indesign, After Effects, Dreamweaver and Flashbuilder), which costs $2600 to buy retail. The package also includes a 20 GB free storage space for storing files.
While this is not a typical SaaS model, and the applications still reside on client’s computers as in previous editions, this is a big step for Abobe, and the pricing model could be a disruptive influence in the sector.
What does Creative Cloud provide?
Easy storage and sharing on the cloud: While I could use Dropbox to store and load .PSDs (Photoshop), .AI (Illustrator) and .INDD (Indesign) files, Creative Cloud allows me to directly sync and share from the creative tools, apart from adding comments on the design. Also, there is a big storage space (20 GB) available that is needed for storing the resource-hungry creative files.
Subscription-based pricing model: Instead of paying $2600 upfront, the subscription model allows you to pay an ongoing payment of $50 (if taking a yearly contract) and could set the trend in creative software.
Getting updates and newer products in smaller chunks: Adobe releases a new version of its Creative Suite every 12–18 months. However, this way, they will be able to issue incremental releases.
Ability to install the product on two computers (one primary and one backup): In previous releases, Adobe provided the option to install a backup as long as it ran the same OS platform (Windows or Mac) as the primary installation. Now, that restriction has been removed.
Why is Creative Cloud significant?
I believe that Creative Cloud is important for the following reasons:
- It is the first big venture by Adobe in cloud computing. The company has not made big ventures beyond the desktop, and its new strategy could be interesting. While this is already a good start, Adobe wants to take Creative Cloud further and build it into a social network where artists can share work and professionals can comment on it.
- While the subscription-based pricing model has already become the norm in productivity tools and project management (with the likes of 37 signals and Salesforce.com), creative tools are still firmly attached to the desktop world. Now, this venture brings mainstream creative and artistic tools to the cloud. While the application is not fully run from the cloud, it signals a first step that could revolutionize the cloud industry yet further.
The subscription-based software pricing model is something that is still in its infancy, and we are eager to see if Adobe’s entry will quicken adoption. Although a large percentage consumers are unhappy with the pricing, I believe that this could set the trend in computing.
Taming Big Data [A Big Data Infographic]
Big Data can be a beast. Data volumes are growing exponentially.The types of data being created are likewise proliferating. And the speed at which data is being created – and the need to analyze it in near real-time to derive value from it – is increasing with each passing hour.
But Big Data can be tamed. We’ve got living proof. Thanks to new approaches for processing, storing and analyzing massive volumes of multi-structured data – such as Hadoop and MPP analytic databases — enterprises of all types are uncovering new and valuable insights from Big Data everyday.
Leading the way are Web giants like Facebook, LinkedIn and Amazon. Following close behind are early adopters in financial services, healthcare and media. And now it’s your turn. From marketing campaign analysis and social graph analysis to network monitoring, fraud detection and risk modeling, there’s unquestionably a Big Data use case out there with your company’s name on it.
We here at Wikibon are excited to present this compelling Big Data infographic, which we hope will help you better understand how your peers are applying Big Data today and inspire you tame the Big Data beast yourself. Check out videos, market forecasts and deep research at on our curated page about Big Data.
Blogged from: http://wikibon.org/blog/taming-big-data/
FUSE-based file system backed by Amazon S3.
S3fs is a FUSE filesystem that allows you to mount an Amazon S3 bucket as a local filesystem. It doesn’t store anything on the Amazon EC2, but user can access the data on S3 from EC2 instance, as if a network drive attached to it.
S3fs-fuse project is written in python backed by Amazons Simple Storage service. Amazon offers an open API to build applications on top of this service, which several companies have done, using a variety of interfaces (web, rsync, fuse, etc).
Best Practices- Installing CentOS VM on
– Part 3
ADDING EXTERNAL HDD AND FORMATTING IT
In this continuation series (PART 3), I am going to show you how to add a additional virtual Hard Drive to the CentOS VM, that too without rebooting it..
There are two ways to configure a new disk drive into a CentOS system:
- One very simple method is to create one or more Linux partitions on the new drive, create Linux file systems on those partitions and then mount them at specific mount points so that they can be accessed. This is the approach that will be covered in this tutorial.
- Another approach is to add the new space to an existing volume group or create a new volume group.
When CentOS is installed using the default disk configuration layout, a volume group is created and called VolGroup00. Within this volume group are two logical volumes named LogVol00 and LogVol01 that are used to store the / file system and swap partition respectively. By configuring the new disk as part of a volume group we are able to increase the disk space available to the existing logical volumes. Using this approach we are able, therefore, to increase the size of the / file system by allocating some or all of the space on the new disk to LogVol00. This topic will be discussed in detail in Adding a New Disk to a CentOS Volume Group and Logical Volume.
First, we will add the Hard Disk to the VM…
Select your virtual machine, as you can see from the above PIC selected the virtual machine. Next press the “Settings‘ to open the Virtual Machine Settings dialog.
From the “Virtual Machine Settings” dialog select the “Add…” button at the bottom of the screen. From this dialog you can also modify how much memory you dedicate to the machine when it boots.
Next we will walk through the “Add Hardware Wizard” the process makes it very simple to add additional hardware to a predefined virtual machine. From this screen we can see the many types of hardware we can add to a virtual machine. You can emulate just about any piece of hardware that one can expect in a modern operating system. It definitely makes testing with different configurations and devices much easier. For our example we want to select “Hard Disk” and then select the “Next >” button.
In the next screen we see the three options for adding a new disk. We can “Create a new virtual disk“, this will create a brand new disk on the guest operating system.
The second option, “Use an existing virtual disk“, allows you to mount a disk from another virtual machine.
The last option is to “Use a physical disk“, this allows you to mount a local physical disk to the operating system. This option is akin to NFS mounting a drive to a virtual machine. To add a new disk we select the “Create a new virtual disk” option and select the “Next >” button.
Next we want to select the type of disk. So in this step we want to select “SCSI (Recommended)” and the “Next >” button.
Now we want to set the size of the disk we are creating.
One of the nice features of VMware is that you don’t have to allocate all of the disk when you create it. So if you create a 40 GB disk it doesn’t have to take it all right away, the disk will grow as your virtual machine needs it. I will say this is a big performance hit you take when the disk has to extend, but for most applications its OK.
Also, I will warn that if the virtual disk grows and there is no physical disk left on the host operating system you will see a catastrophic failure and in most cases both the host and guest operating systems lock up and become unusable. (Don’t say I didn’t warn you !)
Lastly, you can split the files into 2GB sizes, while this isn’t necessary, it just makes all the disks much easier to manage and move around. For this step we want to set our disk size (12 GB in this case), I chose to allocate the disk space right now.. 10 GB in size and Split disk into 2 GB files.
This is actually pretty simple in that you decide what you want to physically call the disk and where to put it. .vmdk is the extension for VMware virtual disks. After we name the disk we can select the “Finish” button which adds the disk to the virtual machine.
So now we can see that the new disk has been added to the “Virtual Machine Settings” within the selected virtual machine. From here the disk acts just like it would if you added a new disk to a standalone server. So we select the “OK” button to continue.
After we’ve logged in and accessed a terminal window as root (or another user with root/sudo priveledges) we first want to run fdisk -l to display list of partitions.
In Linux the first SCSI drive is sda, the second sdb, the third sdc, etc. since this was the second SCSI drive we added to the system, the device is known as /dev/sdb
As shown above, the new hard drive has been assigned to the device file/dev/sdb. At this point the drive has no partitions shown (because we have yet to create any).
The next step is to create one or more Linux partitions on the new disk drive. This is achieved using the fdisk utility which takes as a command-line argument the device to be partitioned:
The basic fdisk commands you need are:
- m – print help
- p – print the partition table
- n – create a new partition
- d – delete a partition
- q – quit without saving changes
- w – write the new partition table and exit
In order to view the current partitions on the disk enter the p command:
As we can see from the above fdisk output the disk currently has no partitions because it is a previously unused disk.
The next step is to create a new partition on the disk, a task which is performed by entering n (for new partition) and p (for primary partition):
In this example we only plan to create one partition which will be partition 1.
Next we need to specify where the partition will begin and end. Since this is the first partition we need it to start at cylinder 1 and since we want to use the entire disk we specify the last cylinder as the end.
Note that if you wish to create multiple partitions you can specify the size of each partition by cylinders, bytes, kilobytes or megabytes.
The next option is t. t is used to replace the type of the current partition.
You can list the different types of Partitions by pressing L as shown below.. this will list down all the available partition types..
for our example, we need a Linux Partition, i.e. number 83.. similarly, if you want to create a SWAP out of this new HDD, then the number would be 82 (Linux/ Swap). Type 83 and press Enter key to continue
Now that we have specified the partition we need to write it to the disk using the w command:
If you do a fdisk -l again, you will see the newly created Disk with the Partition table allocated as shown below.
The next step is to create a file system on our new partition.
We now have a new disk installed, it is visible to CentOS and we have configured a Linux partition on the disk.
The next step is to create a Linux Filesystem system on the partition so that the operating system can use it to store files and data. The easiest way to create a file system on a partition is to use the mkfs.ext3 utility which takes as arguments the label and the partition device:
Now that we have createda new file system on the Linux partition of our new disk drive we need to mount it so that it is accessible to the CentOS system and its users.
In order to do this we need to create a mount point. A mount point is simply a directory or folder into which the file system will be mounted. For the purposes of this example we will create a /extra in root (/) directory to match our file system label (although it is not necessary that these values match): mkdir /extra
The file system may then be manually mounted using the mount command: mount /dev/sdb1 /extra
In order to set up the system so that the new file system is automatically mounted at boot time an entry needs to be added to the /etc/fstab file.
The following example shows an fstab file configured to automount our /extra partition:
With the appropriate configuration line added to the fstab file, the file system will automatically mount on the next system restart.
Best Practices- Installing CentOS VM on
– Part 2
INSTALLING VMware TOOLS
in PART 1, we saw how to effectively install CentOS 5.8 on VMware Workstation…
in this PART, we will install VMware Tools in the VM..
NOTE: If you had selected Easy Install during the install process from Workstation, then this step can be avoided as Easy Install installs VMware-Tools automatically.. still if you ever want, you can install VMware-Tools anytime you want…
Benefits of VMware Tools
The followings are benefits of VMware Tools on your Guest OS:
- A set of VMware device drivers:
These drivers include:
- Graphics Performance: VMware Tools installs a SVGA driver that increases vide0 refresh.
- Efficient Memory Allocation: A memory control driver that is installed with VMware Tools provides increased efficiency in memory allocation.
- Accelerated Mouse Driver: For increased mouse responsiveness.
- Optimized SCSI Driver: A Bus Logic SCSI driver provides faster I/O performance for some guest OS.
VMware Tools control panel:
The Tools control panel lets you modify settings.
- Run Scripts: Scripts can be defined to run after particular events such as the VM starting up or shutting down.
- Shrink the Virtual Disk: By reducing the amount of disk space a VM’s virtual disk takes up by removing empty unused space will lead to more usable storage on your hosts volume.
Connect External Devices:
- You can connect external media devices such as a floppy (who uses those anymore) or CD/DVD drive. These appear as a virtual device when presented within the guest
To summarize,Installing VMware Tools eliminates or improves the following issues:
- Low video resolution
- Inadequate color depth
- Incorrect display of network speed
- Restricted movement of the mouse
- Inability to copy and paste and drag-and-drop files
- Missing sound
Installing VMware Tools within X
Select VM > Install VMware Tools in your Workstation menu as shown:
When you choose VM > Install VMware Tools from the VMware Workstation menu, VMware Workstation temporarily connects the virtual machine’s first virtual CD-ROM drive to the ISO image file that contains the VMware Tools installer for your guest operating system and you are ready to begin the installation process.
Double-click the VMware Tools CD icon on the desktop.
Note: In some Linux distributions, the VMware Tools CD icon may fail to appear when you install VMware Tools within an X windows session on a guest.
Extract the VMwareTools.tar to the Desktop as shown below:
In an X terminal, logged in as root (su -), configure VMware Tools by navigating to the path where we extracted the VMwareTools.tar contents.. in this case, its /root/Desktop/vmware-tools-distrib/
Once there, simply execute the vmware-install.pl file as shown below:
Respond to the questions the installer displays on the screen. Press Enter to accept the default value.
Note: Be sure to respond yes when the installer offers to run the configuration program.
Note: Some guest operating systems require a reboot for full functionality.
Installing VMware Tools from the Command Line with the Tar Installer
The first steps are performed on the host, within Workstation menus:
1. Power on the virtual machine.
2. After the guest operating system has started, prepare your virtual machine to install VMware Tools.
- Choose VM > Install VMware Tools.
The remaining steps take place inside the virtual machine.
3. As root (su -), mount the VMware Tools virtual CD-ROM image, change to a working directory (for example, /tmp), uncompress the installer, then unmount the CD-ROM image.
Note: Some Linux distributions automatically mount CD-ROMs. If your distribution uses automounting, do not use the mount and umount commands below. You still must untar the VMware Tools installer to /tmp.
Some Linux distributions use different device names or organize the /dev directory differently. If your CD-ROM drive is not /dev/cdrom or if the mount point for a CD-ROM is not /mnt/cdrom, you must modify the following commands to reflect the conventions used by your distribution.
- mount /dev/cdrom /mnt/cdrom
- cd /tmp
Note: If you have a previous installation, delete the previous vmware-distrib directory before installing. The default location of this directory is
4. Untar the VMware Tools tar file:
- tar zxf /mnt/cdrom/VMwareTools-5.0.0-<xxxx>.tar.gz
- umount /dev/cdrom
Where <xxxx> is the build/revision number of the VMware Workstation release.
Note: If you attempt to install a tar installation over an rpm installation — or the reverse — the installer detects the previous installation and must convert the installer database format before continuing.
5. Run the .tar VMware Tools installer:
- cd vmware-tools-distrib
Respond to the configuration questions on the screen. Press Enter to accept the default value.
6. Log off of the root account.
7. Start X and your graphical environment
8. In an X terminal, launch the VMware Tools background application.
- vmware-toolbox &
Note: You may run VMware Tools as root or as a normal user. To shrink virtual disks, you must run VMware Tools as root (su -).
Uninstalling VMware Tools
To remove VMware Tools from your Linux guest operating system, log on as root (su -) and enter the following command:
rpm -e VMwareTools
Best Practices- Installing CentOS VM on
– Part 1
INSTALLING CentOS AS A VIRTUAL MACHINE
In this series of blogs I am going to share my experiences and few best practices when it comes to installing and configuring a LINUX, in this case, a CentOS VM using VMware Workstation 8.
NOTE that these steps can well be used to install any LINUX VM on vSphere platforms as well… just the tuning with respect to the class of OS changes then.. like Debian, or Red-Hat..
Software used in this tutorial:
We are going to create a TYPICAL Virtual Machine
Now there are two ways of installing a VM using VMware Workstation.. first; there is something called as Easy Install which assumes certain criteria for installing the OS and provides a quick way to complete the OS installation.. but i am going to avoid that and install the OS manually. In this way, i can specify the Partitioning tables for my CentOS as I want it.
Select Linux from the Guest OS Menu and CentOS 64-Bit as the Version from the drop down list as shown
Provide a suitable name for your VM.. i usually go with Defaults…
Specify 5 GB as the Disk Capacity.. we don’t want to provide a lot of Disk unnecessarily.. 5 GB is enough for installing any LINUX OS.. We will show you how to attach a HDD to the VM in our PART 3 of the series..
Click Customize Hardware to mount the CentOS ISO File manually and do other bit of “cleaning up” !!
First of all, we do the “clean up“.. just remove unwanted devices like Floppy Drives and Printers (Unless you want to use them for your self !!)
Click on CD-ROM (IDE 1) item, select Use ISO image option and click Browse button to select your recently downloaded *.iso installation file of CentOS as shown below
Once done, the VM is ready to be Powered ON. Select the Power On this Virtual Machine as shown below
When your virtual pc starts, you’ll see welcome screen of installation CentOS. Press ENTER to begin the Installation Process…
If you’ll not be able to see this screen, then restart your virtual machine and click Esc button for Boot menu. And from this menu, select CD-ROM Drive
Click ENTER. The installation will tell you to begin testing of your installation CD or DVD. Select Skip button. It will skip testing your CD media and save your time..
Then you’ll get another welcome screen, you only have to click next button on this screen
Select your language of installation and click Next
Then, select your keyboard configuration to use it for the system you’re installing and click Next
Then, you’ll be warned to create new partitions by erasing ALL DATA on your newly created hard drive. As there’s no information in this drive, you’ll select Yes
We will Create a Custom Layout as in this case we will be able to specify the partitions and provide them sizes and all 🙂
We will first create a SWAP Partition. the size of the SWAP Partition depends on how much RAM is provided to the VM.. in this case, the VM is provided with 1 GB RAM.. hence we are providing SWAP 1000 MB..
Similarly, create a BOOT Partition of 100 MB…
You can provide the remaining 4000MB (approximately) to the ROOT (/) Partition as shown.. The ROOT partition houses the installation of your CentOS System..
After you have created the Partitions, it should look something like this:
NOTE: Three Partitions are created; namely, sda1, sda2 and sda3
You may get a warning saying you have allocated less SWAP space than available. You can ignore this message as anyways, SWAP is only used when the system is almost at crashing point.. 🙂
You can create the GRUB BOOT Loader on the /dev/sda partitions. Leave this for default values and click NEXT
Then, you’ll see Network Configurations screen, don’t change anything and click next. You can change whatever you want after installation, for now, let it continue in this way and click Next
From this screen, select your region and click Next
Then, you’ll be prompted to enter password for root user, enter it and click Next
To install packages that you may want in your OS, you can select them Customizing your installation here. Select Customize now option and click Next
I am not a big fan of having all these extra services running on my VM.. so I simply remove what ever I don’t require.. in this case, Games and Entertainment, Graphics, Office Suite etc are removed from Installation process.
Now, you’re ready for installation. Click next on this screen, the installation will begin
You’ll see how all packages are installing. After a while, your OS will be ready for use
After the installation is completed, click Reboot button to reboot your OS
After reboot, you’ll face with another Welcome screen. There are some steps need to be completed
Click Forward button on the screen.
Here, just disable your Firewall. Do remember that this is only for test purpose, don’t disable your Firewall on a real system! And click forward button
For testing purpose, it would be good to disable SELinux too 🙂
If you need, make your Date and Time configuration and click Forward
If you want to create a new user, you can fill these blanks. You can pass this step too by clicking Forward button
Then Click Finish and the system will reboot
After reboot, CentOS will be opened and you’ll be prompted to enter username (root) and password (root’s password)
Your CentOS is ready to use! Congratulations !!
Till then, keep VMware-ing !!!
NOTE from yoyoclouds: Although I must admit I am a HUGE fan of VMware… but I hold no grudges what so ever against Microsoft… This was taken from a Blog posted on VMware’s Community Site..!!
Sorry Microsoft; Not Only Does vSphere Cost Less to Buy, It Also Costs Less to Operate
Microsoft’s wildly exaggerated claims of providing a less expensive virtualization platform than VMware vSphere have been hard to miss if you’ve spent any time on the web lately. We’ve previously pointed out the flaws in their math and our public Cost Per Application Calculator clearly shows how deploying a virtual infrastructure built with vSphere will cost about the same as one built using Microsoft Hyper-V and System Center (or even much less when vSphere’s proven VM density advantage over Hyper-V is factored in.) Now we’re pleased to share recent independent test results that show how vSphere also delivers dramatically lower operational costs compared to Microsoft.
The acquisition capital expense (CapEx) advantage for vSphere shown by our Cost Per Application Calculator is just part of the Total Cost of Ownership (TCO) that diligent customers will want to evaluate when choosing a virtualization and cloud platform. The other key TCO element to consider is Operational Expenses (OpEx) representing the ongoing costs of administering your virtual infrastructure. To help customers assess the OpEx differences between vSphere and Hyper-V, we enlisted Principled Technologies to run both platforms in their labs and measure the system administrator labor time needed for typical recurring tasks.
Five Typical Datacenter Tasks Tested
Principled Technologies selected five common tasks that any administrator of a virtualized datacenter must regularly perform and they measured the administrator labor time taken to complete each one using both the VMware and Microsoft platforms. The tasks tested were:
- Shifting virtual machines off a host to accommodate physical maintenance
- Adding storage volumes and redistributing virtual disk files across the new storage
- Isolating storage-intensive “noisy neighbor” virtual machines
- Provisioning new hosts
- Performing a non-disruptive disaster recovery failover test
Care was taken to conduct the scenario tasks as realistically as possible using the full capabilities of the latest released versions of the VMware and Microsoft products available at the time of the testing. vCenter Site Recovery Manager was included in the VMware configuration (and the full list SRM license costs were included in the VMware total cost figures.) vSphere 5 delivered a convincing across-the-board win over Microsoft for each task tested: tasks took 78% to 97% less time to complete using vSphere. The time savings provided by the VMware platform arise from the more advanced capabilities built into vSphere and the more efficient and optimized implementation of those features we’ve perfected over our years of focusing purely on delivering the best virtual infrastructure and cloud platform.
It’s important to note that the OpEx dollar savings shown in the chart above derive from only five representative sysadmin tasks. There are many other regular activities performed by administrators of virtualized datacenters, most of which will show similar efficiency advantages for vSphere over less mature and capable alternatives, so customers should expect even greater total OpEx savings from using vSphere.
Scenarios Delivering Big OpEx Wins for vSphere
Contributors to the biggest operational advantages for vSphere were:
- Storage Distributed Resource Scheduler – When more storage is needed, the vSphere administrator can add volumes and let Storage DRS redistribute virtual disk files to the new volumes automatically with no VM downtime. The Hyper-V administrator must manually redistribute VM storage and make arrangements for VM downtime during the operation.
- Concurrent vMotion – The vSphere administrator can complete physical host maintenance much sooner because vMotion maintenance mode evacuations of VMs can proceed concurrently and at a faster rate (see comparative live migration testing results here.) Hyper-V hosts can only handle one live migration at a time, so administrators are tied up with much longer maintenance windows.
- Storage I/O Control – vSphere makes it easy to cap the storage IOPS consumed by each VM to prevent resource hogging by “noisy neighbors.” Hyper-V has no such feature, so administrators can only respond by dedicating storage volumes for misbehaving VMs – a tedious task requiring VM downtime.
- vCenter Site Recovery Manager – SRM fully automates replication of mission-critical VMs to a remote site and failover in case of disaster. Real disasters may be rare, but full-scale DR tests should be regular events. That’s where the automated and non-disruptive DR failover test features in SRM deliver big operational savings. Setting up DR failovers with Hyper-V requires maintenance-intensive scripting to orchestrate VM replications and restarts and to modify VMs for network isolation.
After making their labor time measurements, Principled Technologies then estimated how many times each task would be repeated over the course of a two-year period in a datacenter operating 1,000 virtual machines. Multiplying the cumulative time taken performing each task by the U.S. national average system administrator compensation rate gave a dollar figure for the OpEx savings. As shown below, the final result was an impressive 91% reduction in operational expenses when using vSphere compared to Microsoft Hyper-V and System Center.
Operational Expenses Dominate Total Costs
The impact of the OpEx savings delivered by vSphere are even more significant when you consider that IT operational expenses are typically much larger than capital expenses. In fact, Gartner survey data shows cross-industry IT OpEx spending is almost three times CapEx spending – even more reason to choose a virtualization platform that will save you money with better operational efficiency long after the initial purchase.
The OpEx savings delivered by vSphere were enough to tip the two-year TCO advantage in favor of VMware in the 1,000-VM datacenter that was the baseline for Principled Technologies’ tests. When you can have the clearly superior features provided by vSphere 5 Enterprise Plus Edition together with vCenter Site Recovery Manager at a lower total cost than Microsoft’s best alternative, it’s easy to make the decision to go with VMware.
If you’re running your datacenter on vSphere now, an OpEx win by vSphere probably isn’t surprising. vSphere users benefit from over a decade of optimizations we’ve built into our platform that derive from experience in thousands of production datacenters. What did surprise us was just how large an operational advantage vSphere has over the Microsoft platform. The results are a clear example of why you need to look at more than just a feature checklist and initial price tag when choosing the virtualization platform for your most critical workloads.
Take a look for yourself at the full test report by Principled Technologies here (registration required.)