Load Balancing

UniversityLite, an e-commerce rapid deployment tool I developed in 2016, is load balanced across three Ubuntu 14.04 LTS virtual servers located in New York, NY with one backup server in Webster, NY. The three primary servers are balanced alphabetically by university name. There are 7,000 websites available as subdomains off of the universitylite.com domain (such as rit.universitylite.com). Each subdomain is given resources by the server only if actively being viewed. As seen in the image below, an arbitrary university named ‘David’s University College’ would be hosted on Server1. Because the functions and algorithms to create any of the A-Z universities resides on each ot the servers, the balancing is done for resource balancing, not necessarily content balancing.

UniversityLite Server Balance

Because of the large set of active subdomains, load balancing through DNS would be cumbersome and time consuming as partial wildcards (such as a*.universitylite.com) in a DNS zone are not defined behavior within the RFC. Manually adding a record for rit.universitylite.com, for example, would also require us to add a manual record for the 6,999 other subdomains. Therefor a DNS zone can only practicaly be used with full wildcards such as *.universitylite.com to to one of the universitylite.com servers. As a result, load balancing is handled first by the Apache virtualhost file, and then subsequently by PHP logic once the subdomain is called.

As seen in the function below, we can query the SQL table once a subdomain passed to the UniversityLite webserver. If the subdomain is found as a SQL entry, we can go ahead and assume the server referenced is the correct load balanced server. However, if it isn’t, we a) either can assume it’s load balanced on one of the other servers or b) it is simply not a valid subdomain anywhere on the site. For example, trying to go to gaboldygooky.universitylite.com will fail all load balancing checks and return an error to the browser.

UniversityLite PHP Load Balance Example

In this way, I have created a dynamic load balancing system that allows sites to be added and removed ad-hoc without having to alter DNS. Of course, this is just general domain load balancing, and we must further balance the content itself. There are, afterall, 7,000 different subdomains with uniquely generated HTML, photos, graphs, etc. The easiest way I found to handle this is by having UniversityLite just be one central source of programming code, and to allow other content to be displayed dynamically when required. That is not to say, however, that each site can be dynamically created each time it is viewed, but we can remove some redundancy from the equation.

Below are areas of dialy generated content that CAN be shared between them, assuming the same calculator with description, etc. might be viewed by more than one subdomain. This data alone amounts to about 10GB of freshly generated content for a 24 hour period for the sites on server1. If we did not consolidate this data, this could easily have grown to 23TB of data in just 24 hours on one server.

Directory List of ~/www/universitylite.com/public_html/sites/shared$:
drwxr-xr-x 2 maiolo99 maiolo99 4096 Oct 20 16:49 adpics drwxr-xr-x 7 maiolo99 maiolo99 4096 Oct 20 21:29 amazon_xml drwxr-xr-x 2 maiolo99 maiolo99 4096 Sep 23 18:07 apparel_images drwxrwx--- 2 maiolo99 maiolo99 4096 Sep 18 16:54 avatars drwxrwx--- 2 maiolo99 maiolo99 4096 Jul 26 23:29 holidays drwxr-xr-x 356 maiolo99 maiolo99 20480 Oct 8 00:30 ms_images drwxr-xr-x 2 maiolo99 maiolo99 7966720 Oct 21 16:51 product_images

However, sometimes there is only so much we can do. As seen in the directory listing below, these files are created uniquely for one of the subdomains (mcc.universitylite.com) below. This information is perfectly unique to Monroe Community College, and there is nothing we can really do about it.

Directory Listing of ~/www/universitylite.com/public_html/sites/MCCTextbooks:
./daily_message: MonroeCommunityCollege_20161019_daily.txt


./graphs-2016-09-19:

graph_MCC_1473753800.jpg

graph_MCC_1473813698.jpg

graph_MCC_1474457188.jpg

graph_MCC_1474581272.jpg

graph_MCC_All Instructional Staff Total_1473662381.jpg

graph_MCC_All Instructional Staff Total_1473662723.jpg

graph_MCC_All Instructional Staff Total_1473722731.jpg

graph_MCC_All Instructional Staff Total_1473773589.jpg

graph_MCC_All Instructional Staff Total_1473791287.jpg

graph_MCC_All Instructional Staff Total_1473813691.jpg

graph_MCC_All Instructional Staff Total_1474227934.jpg

graph_MCC_All Instructional Staff Total_1474457249.jpg

graph_MCC_All Instructional Staff Total.jpg

graph_MCC_Assistant Professor_1473662380.jpg

graph_MCC_Assistant Professor_1473662722.jpg

graph_MCC_Assistant Professor_1473722730.jpg

graph_MCC_Assistant Professor_1473773588.jpg

graph_MCC_Assistant Professor_1473791286.jpg

graph_MCC_Assistant Professor_1473813690.jpg

graph_MCC_Assistant Professor_1474227933.jpg

graph_MCC_Assistant Professor_1474457248.jpg

graph_MCC_Assistant Professor.jpg

graph_MCC_Associate Professor_1473662379.jpg

graph_MCC_Associate Professor_1473662722.jpg

graph_MCC_Associate Professor_1473722730.jpg

graph_MCC_Associate Professor_1473773588.jpg

graph_MCC_Associate Professor_1473791286.jpg

graph_MCC_Associate Professor_1473813690.jpg

graph_MCC_Associate Professor_1474227932.jpg

graph_MCC_Associate Professor_1474457248.jpg

graph_MCC_Associate Professor.jpg

graph_MCC_degrees_by_race.jpg

graph_MCC_demographics_1474316106.jpg

graph_MCC_demographics_1474316169.jpg

graph_MCC_demographics_1474316231.jpg

graph_MCC_demographics_1474316252.jpg

graph_MCC_demographics_1474316321.jpg

graph_MCC_demographics_1474316741.jpg

graph_MCC_demographics_1474316754.jpg

graph_MCC_demographics_1474316757.jpg

graph_MCC_demographics_1474316908.jpg

graph_MCC_demographics_1474317796.jpg

graph_MCC_demographics_1474318228.jpg

graph_MCC_demographics_1474318400.jpg

graph_MCC_demographics_1474318536.jpg

graph_MCC_demographics_1474318968.jpg

graph_MCC_demographics_1474318969.jpg

graph_MCC_demographics_1474321987.jpg

graph_MCC_demographics_1474323294.jpg

graph_MCC_demographics_1474329291.jpg

graph_MCC_demographics_1474329312.jpg

graph_MCC_demographics_1474332642.jpg

graph_MCC_female_students_by_age.jpg

graph_MCC_graduates_to_enrolled_by_race_ratio.jpg

graph_MCC_Instructor_1473662380.jpg

graph_MCC_Instructor_1473662722.jpg

graph_MCC_Instructor_1473722731.jpg

graph_MCC_Instructor_1473773588.jpg

graph_MCC_Instructor_1473791286.jpg

graph_MCC_Instructor_1473813690.jpg

graph_MCC_Instructor_1474227933.jpg

graph_MCC_Instructor_1474457249.jpg

graph_MCC_instructor_annual_income.jpg

graph_MCC_Instructor.jpg

graph_MCC_Lecturer_1473662380.jpg

graph_MCC_Lecturer_1473662722.jpg

graph_MCC_Lecturer_1473722731.jpg

graph_MCC_Lecturer_1473773589.jpg

graph_MCC_Lecturer_1473791287.jpg

graph_MCC_Lecturer_1473813691.jpg

graph_MCC_Lecturer_1474227933.jpg

graph_MCC_Lecturer_1474457249.jpg

graph_MCC_Lecturer.jpg

graph_MCC_male_students_by_age.jpg

graph_MCC_Professor_1473662379.jpg

graph_MCC_Professor_1473662721.jpg

graph_MCC_Professor_1473722730.jpg

graph_MCC_Professor_1473773588.jpg

graph_MCC_Professor_1473791286.jpg

graph_MCC_Professor_1473813690.jpg

graph_MCC_Professor_1474227932.jpg

graph_MCC_Professor_1474457247.jpg

graph_MCC_Professor.jpg

graph_MCC_students_by_age.jpg

staff_sallary_report_graph_MCC_0.jpg
./weather:

Rochester_NY_Weather_2016-09-19.txt
./wiki_infobox-2016-10-19:

Bevier20Memorial20Building(BevierMemorialBuilding)_intro_request_phase2.xml

Bevier20Memorial20Building_intro_request_phase1.xml

Bridge20Square20Historic20District(BridgeSquareHistoricDistrict)_intro_request_phase2.xml

Bridge20Square20Historic20District_intro_request_phase1.xml

Campbell-Whittlesey20House(Campbell-WhittleseyHouse)_intro_request_phase2.xml

Campbell-Whittlesey20House_intro_request_phase1.xml

First20Presbyterian20Church20(Rochester20New20York)(FirstPresbyterianChurch(RochesterNewYork))_intro_request_phase2.xml

First20Presbyterian20Church20Rochester20New20York_intro_request_phase1.xml

Hervey20Ely20House(HerveyElyHouse)_intro_request_phase2.xml

Hervey20Ely20House_intro_request_phase1.xml

Immaculate20Conception20Church20(Rochester20New20York)(ImmaculateConceptionChurch(RochesterNewYork))_intro_request_phase2.xml

Immaculate20Conception20Church20Rochester20New20York_intro_request_phase1.xml

Jonathan20Child20House2020BrewsterBurke20House20Historic20District_intro_request_phase1.xml

Jonathan20Child20House2020BrewsterBurke20House20Historic20District(JonathanChildHouseampBrewsterBurkeHouseHistoricDistrict)_intro_request_phase2.xml

Main2020Oak20RIRTR20station_intro_request_phase1.xml

Main2020Oak20(RIRTR20station)(Main)_intro_request_phase2.xml

Monroe20Community20College20_intro_request_phase1.xml

Monroe20Community20College20(MonroeCommunityCollege)_intro_request_phase2.xml

Monroe20Community20College20Sports_intro_request_phase1.xml

Monroe20Community20College20Sports(UniversityofLouisianaatMonroe)_intro_request_phase2.xml

Monroe20Community20College20Team_intro_request_phase1.xml

Monroe20Community20College20Team(UniversityofLouisianaatMonroe)_intro_request_phase2.xml

Monroe20Community20College20Tribunes20_intro_request_phase1.xml

Monroe20Community20College20Tribunes20(ListofcollegeathleticprogramsinNewYork)_intro_request_phase2.xml

Monroe20Community20College20Tribunes_intro_request_phase1.xml

Monroe20Community20College20Tribunes(ListofcollegeathleticprogramsinNewYork)_intro_request_phase2.xml

Monroe20Community20College_intro_request_phase1.xml

Monroe20Community20College(MonroeCommunityCollege)_intro_request_phase2.xml

MonroeCommunityCollege_infobox.xml

Nick20Tahou20Hots_intro_request_phase1.xml

Nick20Tahou20Hots(NickTahouHots)_intro_request_phase2.xml

RochesterNewYork_infobox.xml

RochesterNY_geosearch.json

Third20Ward20Historic20District20Rochester20New20York_intro_request_phase1.xml

Third20Ward20Historic20District20(Rochester20New20York)(ThirdWardHistoricDistrict(RochesterNewYork))_intro_request_phase2.xml

./youtube-2016-10-19: BevierMemorialBuildingRochesterNY_youtube_2016-10-19.txt BridgeSquareHistoricDistrictRochesterNY_youtube_2016-10-19.txt Campbell-WhittleseyHouseRochesterNY_youtube_2016-10-19.txt FirstPresbyterianChurchRochesterNewYork_youtube_2016-10-19.txt geosearch-431012652C-77608488--searchquery-_youtube.txt geosearch-Rochester-NY--searchquery-RochesterNYMusic_youtube.txt HerveyElyHouseRochesterNY_youtube_2016-10-19.txt ImmaculateConceptionChurchRochesterNewYork_youtube_2016-10-19.txt JonathanChildHouseBrewsterBurkeHouseHistoricDistrictRochesterNY_youtube_2016-10-19.txt MainOakRIRTRstationRochesterNY_youtube_2016-10-19.txt MonroeCommunityCollegeSports_youtube_2016-10-19.txt MonroeCommunityCollege_youtube_2016-10-19.txt NickTahouHotsRochesterNY_youtube_2016-10-19.txt ThirdWardHistoricDistrictRochesterNewYork_youtube_2016-10-19.txt

The best we can do with this data is purge it (beyond what we might want to keep for historical purposes). We do this by adding historically usable data into a SQL table for that university, and simply deleting the rest.

SUMMARY

The load balancing techniques used on UniversityLite have allowed us to maintain 7,000 unique websites, on only three servers.

Let’s collaborate!

Contact Me

DAVID MAIOLO

The content on this website, including text, photographs, and any other media, is the property of David Maiolo unless otherwise noted. No part of this website may be reproduced, distributed, or transmitted in any form or by any means, including photocopying, recording, or other electronic or mechanical methods, without the prior written permission of the owner.

DISCLAIMER

The information provided on this website is for general informational purposes only. While I strive to keep the information up-to-date and correct, I make no representations or warranties of any kind, express or implied, about the completeness, accuracy, reliability, suitability, or availability with respect to the website or the information, products, services, or related graphics contained on the website for any purpose. Any reliance you place on such information is therefore strictly at your own risk. In no event will I be liable for any loss or damage including without limitation, indirect or consequential loss or damage, or any loss or damage whatsoever arising from loss of data or profits arising out of, or in connection with, the use of this website. *This website may include links to other websites which are not under the control of David Maiolo. I have no control over the nature, content, and availability of those sites.

Leave a Reply

Leave a Reply Cancel reply

Let’s collaborate!