Scaling Beyond Cloud-only Deployments

Victor Gartvich

An increasing number of companies are using the cloud to fulfill all or some of their computing needs, and this includes companies that provide foundational, platform-level services. Instart Logic is no exception.

From the founding of our company, leveraging the cloud has been a key element of our scalability to meet customer needs. I

n 2011, the first generation of Instart Logic’s production environment was deployed completely in the cloud, on a large geographic scale covering all populated continents.

Over time, we migrated the majority of our network onto physical infrastructure that we own and can fine-tune to specific needs of our customers.

Hybrid of cloud and our “own metal” is best for a mission-critical platform service

Instart Logic still uses some cloud resources for a part of our network for rapid scalability whenever required. Our system architecture therefore is a hybrid of cloud and physical deployments. This blog post recounts our operations team’s experience in running a very high-traffic, computationally-intensive and geographically-distributed service inside and outside the cloud – one that is mission-critical to our customers.

Evolution of server hosting services

Ten years ago many hosting companies located around the globe offered a dedicated server hosting service, whereby a customer could rent a physical server hosted in a remote data center. Normally, it took a long time to order and deploy a new server. The hosting companies asked for long-term contractual commitments, and a majority of them offered their service in only one or two locations in a given country. While even back then it was possible for a SaaS company to deploy a global network using dedicated servers rented from different hosting companies in different countries, the company would invest a lot of effort in managing many vendors with different service standards, contracts, processes, and levels of customer service, etc. It was a lot to keep track of, and not easy to scale the network or change the server configuration, since these operations would require direct involvement of several hosting companies’ technical staffs to perform the required hardware changes in rented servers.

Today many hosting companies have relabeled themselves “cloud” companies but still offer two basic services: virtual servers and dedicated servers. Some companies also offer additional infrastructure services like traffic load balancers, server firewalls, file storage service, and several of the largets players (like Amazon, Rackspace, Internap Cloud) offer their hosting services in multiple US, European and Asian locations. The majority of hosting providers have flexible month-to-month contracts, and some even provide per-minute billing for computing and storage resources. Almost all OS deployment and basic configuration operations are automated using several popular management platforms (like cPanel) for cloud providers, and there are also many commercial server management tools (like RightScale) available from third-party companies. It is easy to change the hardware configuration of deployed virtual servers, and normally you don’t need to employ a highly qualified system administrator to run your cloud infrastructure.

Virtual servers vs. dedicated physical servers

At the same time, virtual server providers are generally more expensive in the long run and under high load/traffic conditions, and provide lower system performance compared to bare metal servers at the same price point. Shared network environments may suffer from unpredictable changes in performance. While shared virtual servers definitely simplify operations for less-than-highly-technical companies which are not focused on system performance, for more performance-oriented companies the lack of control over the hardware and network stack creates a lot of challenges in fine-tuning and troubleshooting possible issues.

Since virtual servers offer much more flexibility in deployment automation, and a much faster response in provisioning new servers and making changes in server virtual hardware configuration, they have an important role to play in toward satisfying customers’ requirements. At the same time dedicated servers provide better performance stability because CPU/RAM/disk server resources are not shared with other virtual servers on the same bare-metal host server.

There are no doubts that the cloud is a perfect place to start for almost any Internet/SaaS SaaS or Infrastructure as a Service company, and Instart Logic followed this path as part of our development strategy. But we crossed our growth milestones and met our forecast demands for service availability, network performance, and bandwidth requirements became more important, we launched the next stage in our infrastructure plan. The key element: we started deploying our own physical infrastructure in a majority of central network hub locations in the US, Europe and APAC regions.

What you can expect from cloud server providers

So where do I want to see the cloud server providers in the future? It would be great to see cloud providers combine the flexibility and convenience of virtual cloud servers with the stability, high performance and manageability of physical computer infrastructure. Cloud providers around the globe could allow infrastructure managers to build a physical network and server platform from a web interface, just like you can now from AWS’s console for a virtual environment. For shared virtual environments, it would be nice to have the ability to set the minimal guaranteed level of performance for all critical system components – CPU, disk I/O, network capacity, etc. – and make the shared environment feel and perform similarly to dedicated server and network hardware.

Eventually, the level of performance stability, manageability and scalability of cloud virtual services will allow the majority of performance-oriented Internet/SaaS companies to switch completely to the cloud. But until that day arrives, we can provide superior web application acceleration to our customers only if we control our own physical servers. For the near term, therefore, Instart Logic will primarily rely on our own physical infrastructure deployed around the globe.