Unlimited hosting – behind the scene

November 3, 2016
03
Nov

All limits & statistics are just an example to show you how hosting providers think. Read your hosting provider package description and term of use!

I was researching for an article on the internet to that explain how does unlimited hosting works but from hosting provider perspective. I found many from the client perspective but not from hosting perspective. Since I had my own hosting company and I was managing everything I can explain how I made my plans unlimited.

My explanation is based on Linux hosting mainly using Cpanel and Cloudlinux. Nowadays, any company that uses Cpanel but don`t use Cloudlinux is a big problem, because they can’t limit user resource usage and one user can destroy their server or made server very slow for all other users. There are more users on one server, depend on hardware power, it can be 500, 1000 or few thousands. One user is actually one Cpanel account, and one Cpanel account can have multiple domains, websites, emails and so on, depend on package restrictions. Mainly this kind of hosting has a single server and additional just for redundant DNS. Is it possible to have one more additional dedicated for MySQL, but let’s keep it simple.

The primary server used for everything is like:

  • E5-2630V3 CPU, total 16 HT cores at 2.40 GHz
  • 32 GB RAM
  • 8x 1TB Raid 10 disks, entire 4TB disk space
  • 100Mbps symmetric link, maximum throughput is 12.5 MB/s but in practice around 10 MB/s

I wanted to make unlimited disk space or bandwidth package. In the end, I included both, unlimited disk space and unlimited bandwidth.

Unlimited bandwidth

I made my packages with unlimited bandwidth, that is what is it on the package information.

But wait, my server in theory (720 hours 100% usage) can make maximum 29TB monthly bandwidth, but in practice much less. Have in my mind that also my network speed can be maximum ~10 MB/s at a given time, I needed to limit client throughput speed. That is not noted in hosting packages, not even noted in terms of use on any other hosting provider. So let`s limit our users:

  1. It can use 2MB/s for 1 minute.
  2. If they made more than 120MB in given minute, limit to 1MB/s for next 5 minutes.
  3. If they made 300MB in 5-minute interval, limit to 500KB/s

If some user is using too much bandwidth, speed will drop to 500KB/s so in theory maximum monthly usage that user can make is ~1TB, but in practice much lower. Also if his site is using that kind of bandwidth power, it will be extremely slow since many visitors are waiting to open their site, but the site has only 500KB/s total for all visitors. This limitation also can be made per account, so all sites under the same account share the same speed.

In addition, they are terms of use like:

  • You can’t stream a video
  • You can’t embed a video
  • You can’t provide huge download files
  • Insert your new rule to suspend your next abuser

Datacenter network price can vary very much depend on which network provider they use. Prices can be even 10 times more or less, so for some hosting company maybe 1TB is the red line for another one maybe 100GB is the red line. Imagine you have everything in terms of a data center, hardware, software etc.. you just need an internet connection to start your hosting company. Request quota for 10Gbps symmetric internet link from Level 3 and Hurricane Electric as global internet service providers to see the difference in price. But that is another topic, in short, the quality network is expensive.

Unlimited Disk Space

I used enterprises disks but wanted to be cheap while providing some decent performance. Keep in mind that also OS cache the recently used files, so this speed need to be normal. In the end, it`s shared hosting, most low-end type of hosting!

I was using Seagate 1TB Enterprise HDD SATA 6Gb/s which is around 110$. The whole raid is around 900$ and gives me 4TB of disk space or price/space is 200$ per TB. That is cheap; you can have tons of users on 1TB while costing you 200$. But throughput speed and IOPS are very limited. Let`s design our unlimited disk space.

  1. User account for all sites can have maximum 1,024 KB/s throughput speed.
  2. User account for all sites can have a maximum of 200,000 inodes. (number of files)
  3. User account for all sites can have a maximum of 20 IOPS. (disk operations per second)

Keep in mind; there are additional restrictions that almost every shared web hosting have terms like:

  • You cannot backup/archive your private files like videos or images to your web disk space. It can be used only for your website content.
  • Maximum file is 5GB
  • Insert your new rule to suspend your next abuser

It’s really hard to calculate this usage since all limits are kind related. But let`s explain as much as I can. I think the most important factor here is a number of inodes and terms of use that users can’t store private files. Every file on your account is calculated as inode, including everything even received email, sent email, attachment, cached page, log, image and so on, so you can have a maximum of 200,000 files.

I developed an automated script to check each user account and to compare with others accounts and if his number of size / used inodes if huge, I was manually checking him. My experience tells me that no more than 100GB can be used in a normal way with 200,000 inodes, so my script also was checking if someone other is using more than 100GB, maximum file size, etc..

100GB = 102400MB / 200,000 inodes = 0.5MB per file.

Here are the statistics from top 10 disk usage accounts:

Since CPanel do not provide inodes usage in their GUI, I was manually checking.

  • 28024 MB –   94402 inodes
  • 24093 MB – 190718 inodes
  • 22812 MB –   35418 inodes
  • 22677 MB – 104306 inodes
  • 22061 MB – 128877 inodes
  • 21402 MB – 102082 inodes
  • 20508 MB –   17680 inodes
  • 19309 MB – 402622 inodes (custom package)
  • 18206 MB – 131811 inodes
  • 17925 MB –   58951 inodes

Interesting statistics, those in bold looks suspicious. Let’s manually check how are they use the disk space.

20508 MB –   17680 inodes

Since this user has a low number of inodes, he must have few large files. Let’s check top 10 largest files.

731664 /newhome/anonymous/public_html/wp-content/uploads/backupbuddy_backups/backup-anonymous-2016_05_10-04_23am-full-yphcedespj.zip
731652 /newhome/anonymous/public_html/wp-content/uploads/backupbuddy_backups/backup-anonymous-2016_05_03-03_47am-full-1zawov61qy.zip
731644 /newhome/anonymous/public_html/wp-content/uploads/backupbuddy_backups/backup-anonymous-2016_04_26-03_50am-full-nrvx02txiy.zip
731632 /newhome/anonymous/public_html/wp-content/uploads/backupbuddy_backups/backup-anonymous-2016_04_19-04_16am-full-re0xkbh1z8.zip
731620 /newhome/anonymous/public_html/wp-content/uploads/backupbuddy_backups/backup-anonymous-2016_04_12-03_47am-full-dsb9aj9tp4.zip

 

Interesting is that site is WordPress and he has a backup plugin that keeps his backups for last six months every day, and one backup file is up to 750MB. Even the most critical websites do not have a daily backup for last six months and the problem to be more huge, it will create backups for forever, until system admin does not warn him and he doesn’t fix it. So what is next?

  1. Warn the user to fix it in X days.
  2. If the user does not fix it, suspend him.
  3. Create new term of service rule. “Your site can include maximum 30 backups”.

You just can’t use that 5$ monthly hosting for storing your private files or spending 5000$ on ads. Everything has USE CASE! In the end, for just 5$ you are getting ALL-IN-ONE good starting point and as you grow, get some better hosting type.