| Message ID | 20210119093746.293342-1-pbonzini@redhat.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | [qemu-web] deploy to qemu-project.org from GitLab CI | expand |
On Tue, Jan 19, 2021 at 10:37:46AM +0100, Paolo Bonzini wrote: > Currently, the website is rebuilt on qemu-project.org using > an update hook. We can reuse instead the Jekyll output of > GitLab's CI. Are there any files present on the qemu-project.org webroot other than the published output of Jekyll ? QEMU had the good sense to use sub-domains git.qemu-project.org and download.qemu-project.org for other content. So maybe you just turn the root qemu-project.org and www.qemu-project.org domains into CNAME records pointing to the gitlab pages site ? > diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml > index 5fa3041..08b6d20 100644 > --- a/.gitlab-ci.yml > +++ b/.gitlab-ci.yml > @@ -1,6 +1,11 @@ > > +stages: > + - build > + - update > + > pages: > image: centos:8 > + stage: build > cache: > paths: > - vendor > @@ -14,3 +19,26 @@ pages: > artifacts: > paths: > - public > + > +deploy: > + image: centos:8 > + stage: update > + needs: > + - job: pages > + artifacts: true > + before_script: > + - dnf install -y openssh-clients rsync > + - eval $(ssh-agent -s) > + - cat "$SSH_PRIVATE_KEY_FILE" | tr -d '\r' | ssh-add - > + - mkdir -m700 -p ~/.ssh > + - - '[[ -f /.dockerenv ]] && echo -e "Host *\n\tStrictHostKeyChecking no\n\n" >> ~/.ssh/config' > + script: > + - ssh $SSH_DEPLOY_DESTINATION "cd /var/www/qemu-project.org && mkdir new && rsync -avz root/ new" Why copy the old root into the new root ? This means that any time we delete a file in git, it will never be removed from the live webroot. > + - rsync -avz public/ $SSH_DEPLOY_DESTINATION:/var/www/qemu-project.org/new > + - ssh $SSH_DEPLOY_DESTINATION "cd /var/www/qemu-project.org && rm -rf old && mv root old && mv new root" Regards, Daniel
On 19/01/2021 10.37, Paolo Bonzini wrote: > Currently, the website is rebuilt on qemu-project.org using > an update hook. We can reuse instead the Jekyll output of > GitLab's CI. > > To do so, a new user qemu-deploy has been created on qemu.org. > The private key is stored into a file variable SSH_PRIVATE_KEY_FILE > (be careful to include the trailing newline after "---END OPENSSH > PRIVATE KEY---"!). > > Signed-off-by: Paolo Bonzini <pbonzini@redhat.com> > --- > .gitlab-ci.yml | 28 ++++++++++++++++++++++++++++ > 1 file changed, 28 insertions(+) > > diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml > index 5fa3041..08b6d20 100644 > --- a/.gitlab-ci.yml > +++ b/.gitlab-ci.yml > @@ -1,6 +1,11 @@ > > +stages: > + - build > + - update There is already "stages" section a the beginning of the yml file, so I'd suggest to extend that instead. Thomas
On 19/01/21 10:58, Thomas Huth wrote: > On 19/01/2021 10.37, Paolo Bonzini wrote: >> Currently, the website is rebuilt on qemu-project.org using >> an update hook. We can reuse instead the Jekyll output of >> GitLab's CI. >> >> To do so, a new user qemu-deploy has been created on qemu.org. >> The private key is stored into a file variable SSH_PRIVATE_KEY_FILE >> (be careful to include the trailing newline after "---END OPENSSH >> PRIVATE KEY---"!). >> >> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com> >> --- >> .gitlab-ci.yml | 28 ++++++++++++++++++++++++++++ >> 1 file changed, 28 insertions(+) >> >> diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml >> index 5fa3041..08b6d20 100644 >> --- a/.gitlab-ci.yml >> +++ b/.gitlab-ci.yml >> @@ -1,6 +1,11 @@ >> +stages: >> + - build >> + - update > > There is already "stages" section a the beginning of the yml file, so > I'd suggest to extend that instead. Are you sure? This is the qemu-web CI. Paolo
On 19/01/21 10:53, Daniel P. Berrangé wrote: > On Tue, Jan 19, 2021 at 10:37:46AM +0100, Paolo Bonzini wrote: >> Currently, the website is rebuilt on qemu-project.org using >> an update hook. We can reuse instead the Jekyll output of >> GitLab's CI. > > Are there any files present on the qemu-project.org webroot > other than the published output of Jekyll ? QEMU had the > good sense to use sub-domains git.qemu-project.org and > download.qemu-project.org for other content. So maybe you > just turn the root qemu-project.org and www.qemu-project.org > domains into CNAME records pointing to the gitlab pages site ? There are other files in the webroot at https://www.qemu.org/docs/master/, even though they're handled server side with the httpd Alias directive. Right now they are built outside CI, from a cron job, but the plan is to do the same as qemu-web and deploy them during the GitLab CI run. >> diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml >> index 5fa3041..08b6d20 100644 >> --- a/.gitlab-ci.yml >> +++ b/.gitlab-ci.yml >> @@ -1,6 +1,11 @@ >> >> +stages: >> + - build >> + - update >> + >> pages: >> image: centos:8 >> + stage: build >> cache: >> paths: >> - vendor >> @@ -14,3 +19,26 @@ pages: >> artifacts: >> paths: >> - public >> + >> +deploy: >> + image: centos:8 >> + stage: update >> + needs: >> + - job: pages >> + artifacts: true >> + before_script: >> + - dnf install -y openssh-clients rsync >> + - eval $(ssh-agent -s) >> + - cat "$SSH_PRIVATE_KEY_FILE" | tr -d '\r' | ssh-add - >> + - mkdir -m700 -p ~/.ssh >> + - - '[[ -f /.dockerenv ]] && echo -e "Host *\n\tStrictHostKeyChecking no\n\n" >> ~/.ssh/config' >> + script: >> + - ssh $SSH_DEPLOY_DESTINATION "cd /var/www/qemu-project.org && mkdir new && rsync -avz root/ new" > > Why copy the old root into the new root ? This means that any time we delete > a file in git, it will never be removed from the live webroot. To speed up the rsync below. The website is small, but it doesn't hurt. However, you're right that "--delete" must be added to the next line. Paolo >> + - rsync -avz public/ $SSH_DEPLOY_DESTINATION:/var/www/qemu-project.org/new >> + - ssh $SSH_DEPLOY_DESTINATION "cd /var/www/qemu-project.org && rm -rf old && mv root old && mv new root" > > > > Regards, > Daniel >
On 19/01/2021 11.00, Paolo Bonzini wrote: > On 19/01/21 10:58, Thomas Huth wrote: >> On 19/01/2021 10.37, Paolo Bonzini wrote: >>> Currently, the website is rebuilt on qemu-project.org using >>> an update hook. We can reuse instead the Jekyll output of >>> GitLab's CI. >>> >>> To do so, a new user qemu-deploy has been created on qemu.org. >>> The private key is stored into a file variable SSH_PRIVATE_KEY_FILE >>> (be careful to include the trailing newline after "---END OPENSSH >>> PRIVATE KEY---"!). >>> >>> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com> >>> --- >>> .gitlab-ci.yml | 28 ++++++++++++++++++++++++++++ >>> 1 file changed, 28 insertions(+) >>> >>> diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml >>> index 5fa3041..08b6d20 100644 >>> --- a/.gitlab-ci.yml >>> +++ b/.gitlab-ci.yml >>> @@ -1,6 +1,11 @@ >>> +stages: >>> + - build >>> + - update >> >> There is already "stages" section a the beginning of the yml file, so I'd >> suggest to extend that instead. > > Are you sure? This is the qemu-web CI. Never mind, I looked at the wrong yml file :-( Thomas
diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index 5fa3041..08b6d20 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -1,6 +1,11 @@ +stages: + - build + - update + pages: image: centos:8 + stage: build cache: paths: - vendor @@ -14,3 +19,26 @@ pages: artifacts: paths: - public + +deploy: + image: centos:8 + stage: update + needs: + - job: pages + artifacts: true + before_script: + - dnf install -y openssh-clients rsync + - eval $(ssh-agent -s) + - cat "$SSH_PRIVATE_KEY_FILE" | tr -d '\r' | ssh-add - + - mkdir -m700 -p ~/.ssh + - - '[[ -f /.dockerenv ]] && echo -e "Host *\n\tStrictHostKeyChecking no\n\n" >> ~/.ssh/config' + script: + - ssh $SSH_DEPLOY_DESTINATION "cd /var/www/qemu-project.org && mkdir new && rsync -avz root/ new" + - rsync -avz public/ $SSH_DEPLOY_DESTINATION:/var/www/qemu-project.org/new + - ssh $SSH_DEPLOY_DESTINATION "cd /var/www/qemu-project.org && rm -rf old && mv root old && mv new root" + only: + refs: + - master + variables: + - $SSH_PRIVATE_KEY_FILE + - $SSH_DEPLOY_DESTINATION
Currently, the website is rebuilt on qemu-project.org using an update hook. We can reuse instead the Jekyll output of GitLab's CI. To do so, a new user qemu-deploy has been created on qemu.org. The private key is stored into a file variable SSH_PRIVATE_KEY_FILE (be careful to include the trailing newline after "---END OPENSSH PRIVATE KEY---"!). Signed-off-by: Paolo Bonzini <pbonzini@redhat.com> --- .gitlab-ci.yml | 28 ++++++++++++++++++++++++++++ 1 file changed, 28 insertions(+)