Virtual Labs is a mission mode project initiated by the Ministry of Human Resources and Development (MHRD). The objective of this project is to provide laboratory learning experience to the students who do not have access to adequate laboratory infrastructure. Currently there are around 90 labs developed by various institutes in the consortium. A streamlined software development life cycle process followed for the development of these labs ensures high quality labs. This document defines the hosting process to be followed by the developers (open source community) of the Virtual Labs project.
This document defines the experiment on-boarding and hosting process that should be followed by the developers of the lab while requesting the hosting of their lab on AWS. The roles and responsibilities of the various parties involved in the hosting process is also discussed in detailed in the document.
A well defined experiment hosting and on-boarding process will help maintain a consistent user experience and enable experiment-authors to focus on the content. Consolidated information regarding all the deployments will also facilitate reporting.
The target audience for this document is the hosting team at CPE, IIITH and all the lab authors and owners who want to avail the hosting service provided by Central Platform Engineering Team (CPE), IIITH team.
Virtual Labs hosting process will require certain pre-requisites to be met by the experiments that need to be hosted. The following paragraphs define terminologies used during the hosting process.
Hosting is a service provided by the Central Platform Engineering (CPE) which includes the following:
Following are not part of the hosting service:
Hosting unit is the code base/repository that will hosted in its enirety during the hosting process. It will not be possible for a part of a hosting unit to be hosted separately. A hosting unit will correspond to one public repository.A hosting unit could be a stand alone experiment or could be a lab consisting of a set of related experiments.
Each repository, corresponding to a hosting unit, will have one designated primary owner. This owner will be the single entity responsible for all the code that resides in the repository. The owner will also be the final approval authority for a hosting request. The owner should ensure that the master branch of the repository only contains the code that needs to be hosted. The latest code in the master branch should represent the hosted experiment.
The primary repository owner does not have to be a person. It may be any uniquely identifiable entity. From the perpective of Virtual Labs hosting team, this entity will the single point of contact for all clarifications. The hosting team will not be responsible for any confusions arising due to shared accessibility of the repo owner entity.
A repository owner or a developer who raises a hosting request.
The repository owner will be responsible for tagging each merge to the master branch. The owner will have to follow Semantic Versioning . In short, each version is a combination of three numbers (MAJOR.MINOR.PATCH) separated by dots. The changes to these numbers represent the following:
All Virtual Labs hosting request should specify a tag.
Each hosting unit should have a well defined build process. The hosting team will treat the build process as a black box action. This action will be triggerred to generate the artifacts to deploy. The repository owner should ensure that the build process is well documented and functioning as desired before raising a hosting request.
The hosting process will result in a deployment with an accessible url. The responsibility of testing the deployments will lie with the repository owner. The hosting team will be responsible for providing the link to the deployment. The hosting team will not provide any testing environment. The repository owner will be responsible for ensuring that the hosting unit works as expected in their own specific testing environments.
Virtual Labs hosting process consists of two steps -
Each hosting unit that needs to be hosted by the Virtual Labs hosting team, must follow an onboarding process. The repository owner of the hosting unit will need to raise an issue of type on-boarding request in the engineers-forum. The issue will need to filled with the following pieces of information :
On-Boarding Request -
Repository owner will be responsible for keeping the above information current and intimating the hosting team of any updates/changes as and when needed.
On receipt of an onboarding request the hosting team will verify the data and acknowledge the same as part of the request.
The hosting team will be responsible for storing the this information at a central location and making it available for all stakeholders.
A repository owner/developer (requester ) will have to raise an issue of type hosting request in the GitHub repository engineers-forum under Virtual-Labs organisation for hosting of a hosting unit. The issue will need to filled with the following pieces of information :
A hosting request will be tied to this information. This issue will be a single source of truth for that hosting request. All communication related to the hosting will be recorded on the raised hosting request.
The hosting request will go through the following lifecycle:
At any given time a hosting issue should be marked with only one of the following labels. To change the label of an issue, the current label of the issue should be unchecked and the new label should be checked.
Hosted : This label indicates that the hosting request has been successful and the hosted url has been shared in the hosting request issue. This label is used only by the hosting team.
Failed : This label indicates that the hosting request has not been successful. This label is used only by the hosting team.
Reopened : This label is used by the requester to indicate that the hosting url provided on a successful hosting of the hosting unit is not passing the validation.
Not Approved : This label indicates that the hosting request issue raised by the requester was not approved by the Repository Owner. This label is used only by the hosting team.
Reverted : This label indicates that the hosting request has been successfully revert to the previous hosted image. This label is used only by the hosting team.
Apart from the above mentioned labels a hosting request issue can either be in a closed or open status as provided by GitHub.
Open : This status indicates that a new hosting request issue has been raised and is waiting for the services of the hosting team. All issues are in this status when they are created on GitHub.
Closed : This status indicates that a hosting request has been serviced by the hosting team and is labelled as Hosted or Reopened or Reverted or Not Approved.
Keeping the deliverables/due date and the current hosting model in mind, the following process will need to be followed for getting a Lab hosted on AWS during the Phase III of the Virtual Labs project ( ending on March 31st 2020). This is as per many discussions with IITB and the meeting on [2020-03-03 Tue].
VLEAD and IITB will follow the process detailed below for the hosting of labs which have been developed with each experiment having its own repository. All the Phase 3 labs and experiments hosted on AWS will have a common UI and report analytics at both lab and experiment level.
IIITH will create lab repo for the lab that is ready ( marked in green) in the 200224_Deliverable_Status_Phase3 sheet shared by IITB and also populate the lab urls in the sheet.
IITB will raise an issue of type Phase 3 Onboarding Request and fill the lab name and lab url ( as populated in the excel sheet by IIITH) and attach the R0 of the lab.
IIITH will tag the head of the master branch of all the experiments in that lab as mentioned in the 200224_Deliverable_Status_Phase3 as v1.0.0
IIITH will populate the received Phase 3 Onboarding Request with experiment details ( name, url, tag, branch and build command) and Owner details and request for approval from IITB . It will be assumed by IIITH that no special hardware or software is required for the running of the experiments. IITB will have to specially state if this is not the case.
IITB will validate the above data and approve the same on the received Phase 3 Onboarding Request .
On approval from IITB , IIITH will host all the experiments and populate the lab landing page information (Introduction, Objective, List of experiments, target audience, course alignment ) from the R0 shared by IITB in the Phase 3 Onboarding Request.
IIITH will host the lab and share the hosted url link as part of the onboarding request issue and seek approval from the github handle of the owner of the lab ( as recorded in the onboarding request)
On approval from the Lab owner, IIITH will share the link with IIT Delhi.
VLEAD and the institute requesting the rehosting of Phase 3 lab while changing the content of the lab will follow the process detailed below-
Institute requesting rehosting will follow the link and locate the Phase III Lab/Experiment(s) OnBoarding Request issue corresponding to the lab that they wish to rehost.
They will comment on the issue with the content that they like to add to the lab pages following the format - Lab Section Name - New content
VLEAD's hosting engineer will host the lab with the requested changes and share the link on the same issue to seek approval from institute requesting rehosting.
Institute requesting rehosting will need to test the hosted link and on their approval the hosted link will be shared with IITD (to be added to vlab.co.in).
VLEAD and the institute requesting the rehosting of Phase 3 lab after changing the content of experiment will follow the process detailed below-
Institute requesting rehosting will make the necessary changes to the experiment content in the experiment repository on GitLab ( IITB server ) or Github (virtual-labs organization) as per the process followed during experiment development. After they make the changes to the experiment sources , they will need to tag it.
Then will then follow the link and locate the Phase III Lab/Experiment(s) OnBoarding Request issue corresponding to the lab containing the experiment that they wish to rehost.
They will comment on the same Phase III Lab/Experiment(s) OnBoarding Request with the following text "Request to rehost the experiment <exp repo link> with the tag <tag number>". They may include more than one experiment ( belonging to the same lab) rehosting request at a time.
VLEAD's hosting engineer will host the experiments and share the link on the same issue to seek approval from them.
Institute requesting rehosting will need to test the hosted link and comment on the same issue stating their approval.
VLEAD and the institute requesting the movement of Phase 3 experiment repositories from IITB Gitlab Server to virtual-labs GitHub organization will follow the process detailed below-
Institute requesting movement will raise an issue to request creation of experiment repositories for the lab.
They will fill the information requested for on the issue and attach R0 ( proposal of the lab)
VLEAD's hosting engineer will create experiment repositories for each of the experiment on the proposal on Github and share the links of the created repositories on the same issue.
Institute requesting movement will populate the dev branches of the created repositories with the source code of the experiments ( same phase 3 structure as defined in the IIT B exp repos) and unit test the experiments locally.
Institute requesting movement will merge the fully tested dev branch to testing branch. This will automatically deploy the experiment along with UI on GitHub pages for testing the complete experiment. The Institute requesting movement will receive an email about the success/failure of the deployment on the email id associated with the github handle provided in the issue.
Institute requesting movement will then merge from a fully tested testing branch to the main branch and tag the commit.
Institute requesting rehosting will then follow the link and locate the Phase III Lab/Experiment(s) OnBoarding Request issue corresponding to the lab that they wish to rehost. If such an issue does not exist they will create will raise an issue of type Phase 3 Onboarding Request. In the issue, for each experiment they will provide the Experiment Name|Experiment Source Repo link|Tag . They will also attach the R0 of the lab to the issue if not already attached.
VLEAD's hosting engineer will host the experiments and the lab and share the link on the same issue to seek approval from them.
Institute requesting momement will need to test the hosted link and comment on the same issue stating their approval.
Steps to be followed by VLEAD's Hosting Engineer
On receiving a request for creation of experiment repositories for the lab, VLEAD's hosting engineer will create the experiment(s) repository from the ph3-exp-template repository.
VLEAD's hosting engineer will create two branches - dev and testing from the main branch in the created experiment repositories.
VLEAD's hosting engineer will add the Github Handle of the experiment developer provided in the request issue to the newly created experiment repository with maintainer access. VLEAD's hosting engineer will also add developer's common email address to github notification center
VLEAD's hosting engineer will comment on the repository creation request providing the links to the created repositories.
The following information will be stored at a central place for a quick reference by the hosting team:
Please follow the link for the Central Hosting Data.
The flow diagram depicting lifecycle of a hosting request is below:
|1.||Raising an On-Boarding Request||Repository owner|
|2.||Tagging the hosting unit release in the||Requester|
|master branch while raising hosting request|
|3.||Testing the application before||Requester|
|making hosting request|
|4.||Raising the hosting request and||Requester|
|providing all required information|
|5.||Seeking approval of the repository owner||Hosting team|
|if the request is raised by someone other|
|than the repository owner.|
|Labelling the hosting request issue as|
|Not Approved if the request is not|
|approved by the repository owner|
|6.||Validating the information provided in the||Hosting team|
|hosting and on-boarding request|
|7.||Running the hosting process||Hosting team|
|8.||Providing the hosted link to the||Hosting team|
|requester and labelling the hosting|
|request issue to Hosted|
|9.||Providing the causes/nature of failure||Hosting team|
|and labelling the hosting request issue to|
|10.||Validating the hosted code||Requester|
|11.||Labelling the hosting request issue as||Requester|
|Reopened if validation is unsuccessful|
|12.||Reverting the hosting unit to previous hosted||Hosting team|
|image if validation fails and|
|labelling the issue as Reverted|
|13.||Changing the status to Closed if the||Requester|
|request is labelled Hosted or Reverted|
|or Not Approved or Failed|
Steps to be followed by VLEAD's Hosting Engineer while creating a service repository
Step 1 On receiving a request for creation of service repository, VLEAD's hosting engineer will create the service repository from the svc-template repository.
Step 2 VLEAD's hosting engineer will create two branches - dev and testing from the main branch in the created service repository.
Step 3 VLEAD's hosting engineer will add the Github Handle of the developer provided in the request issue to the newly created service repository with maintainer access.
Step 4 VLEAD's hosting engineer will comment on the repository creation request providing the links to the created repositories.
Steps to be followed by VLEAD's Hosting Engineer while hosting a service
Why should I use the Hosting Service?
To get more for nothing. Hosting Service provides you a high-availability, high-performance deployment environment. You do not have to spend time, money, people or other resources to ensure that your website is available to your target audience. You can also take advantage of other services, like Analytics, provided by the Central Platform Engineering team and monitor multiple statistics about the users and usage patterns of your Lab.
Do I get better website performance if I use Hosting Service?
Almost all the time. The CPE team works continuously to improve the overall and individual Lab/Experiment performance based on the Analytics data. You can compare the performance of the Labs/Experiments deployed through the hosting service to those deployed elsewhere. If you use the Analytics service integration, the CPE team can individually target your Lab/Experiment for better performance.
Does the CPE team help me improve my Lab performance?
Yes. The CPE team continuously works to improve the infrastructure in order to achieve best performance results for your Lab. The CPE team does not improve your code, however, it can provide you guidelines and actionable performance improvement advice.
Does Hosting Service add a UI to my webpages?
No. Hosting service does not alter your code in any way. As the developer, you are responsible for ensuring that your Lab/Experiment is complete in all respects before you request it to be hosted. Hosting service is strictly limited to its stated objectives.
How can I get Analytics for my experiments?
By working with the CPE team. Analytics service is another service provided by the CPE team. You can work with the CPE team to integrate Analytics service into your Labs. You can also get access to customized dashboards/reports in order to monitor multiple statistics about the users and usage patterns of your Lab.
Does the CPE team test my Lab for correctness as part of hosting?
No. It is not possible for the CPE team to test your Lab because only you understand the right definition of correctness for your Lab/Experiment. As part of hosting, the CPE team ensures that your Lab is up and accessible. You are responsible for ensuring the completeness and correctness of your code and deployment.
Will the CPE team tell me about broken links as part of hosting?
No. Your code is a complete blackbox for the CPE team. You are responsible for ensuring the completeness and correctness of your code and deployment.
This document highlights the hosting process of Virtual Labs from on-boarding to hosting. It aims at collaborating contributions of the developers across the open source platform - GitHub. This streamlined process would help in our strive for excellence in delivering high standard labs.