static: move certs to group, update testing name to static99

Currently we define the letsencrypt certs for each host in its
individual host variables.

With recent work we have a trusted CA and SAN names setup in
our testing environment; introducing the possibility that we could
accidentally reference the production host during testing (both have
valid certs, as far as the testing hosts are concerned).

To avoid this, we can use our naming scheme to move our testing hosts
to "99" and avoid collision with the production hosts.  As a bonus,
this really makes you think more about your group/host split to get
things right and keep the environment as abstract as possible.

One example of this is that with letsencrypt certificates defined in
host vars, testing and production need to use the same hostname to get
the right certificates created.  Really, this should be group-level
information so it applies equally to host01 and host99.  To cover
"hostXX.opendev.org" as a SAN we can include the inventory_hostname in
the group variables.

This updates one of the more tricky hosts, static, as a proof of
concept.  We rename the handlers to be generic, and update the testing
targets.

Change-Id: Id98768e29a06cebaf645eb75b39e4dc5adb8830d
changes/08/848908/4
Ian Wienand 5 months ago
parent b6486ca022
commit e70c1e581c
  1. 73
      inventory/service/group_vars/static.yaml
  2. 73
      inventory/service/host_vars/static01.opendev.org.yaml
  3. 64
      playbooks/roles/letsencrypt-create-certs/handlers/main.yaml
  4. 69
      testinfra/test_static.py
  5. 1
      zuul.d/infra-prod.yaml
  6. 4
      zuul.d/system-config-run.yaml

@ -0,0 +1,73 @@
letsencrypt_certs:
static-opendev-org-main:
- static.opendev.org
- '{{ inventory_hostname }}'
- files.openstack.org
- static.openstack.org
static-ask-openstack-org:
- ask.openstack.org
static-docs-airshipit-org:
- docs.airshipit.org
static-ci-openstack-org:
- ci.openstack.org
static-cinder-openstack-org:
- cinder.openstack.org
static-developer-openstack-org:
- developer.openstack.org
static-devstack-org:
- devstack.org
- www.devstack.org
static-docs-opendev-org:
- docs.opendev.org
static-docs-openstack-org:
- docs.openstack.org
static-docs-starlingx-io:
- docs.starlingx.io
static-eavesdrop-openstack-org:
- eavesdrop.openstack.org
static-glance-openstack-org:
- glance.openstack.org
static-git-airshipit-org:
- git.airshipit.org
static-git-openstack-org:
- git.openstack.org
static-git-starlingx-io:
- git.starlingx.io
static-git-zuul-ci-org:
- git.zuul-ci.org
static-governance-openstack-org:
- governance.openstack.org
static-horizon-openstack-org:
- horizon.openstack.org
static-keystone-openstack-org:
- keystone.openstack.org
static-nova-openstack-org:
- nova.openstack.org
static-meetings-opendev-org:
- meetings.opendev.org
static-planet-openstack-org:
- planet.openstack.org
static-service-types-openstack-org:
- service-types.openstack.org
static-security-openstack-org:
- security.openstack.org
static-specs-openstack-org:
- specs.openstack.org
static-summit-openstack-org:
- summit.openstack.org
static-swift-openstack-org:
- swift.openstack.org
static-releases-openstack-org:
- releases.openstack.org
static-tarballs-opendev-org:
- tarballs.opendev.org
static-tarballs-openstack-org:
- tarballs.openstack.org
static-zuul-ci-org:
- zuul-ci.org
- www.zuul-ci.org
- zuulci.org
- www.zuulci.org
static-gating-dev:
- gating.dev
- www.gating.dev

@ -1,73 +0,0 @@
letsencrypt_certs:
static01-opendev-org-main:
- static.opendev.org
- static01.opendev.org
- files.openstack.org
- static.openstack.org
static01-ask-openstack-org:
- ask.openstack.org
static01-docs-airshipit-org:
- docs.airshipit.org
static01-ci-openstack-org:
- ci.openstack.org
static01-cinder-openstack-org:
- cinder.openstack.org
static01-developer-openstack-org:
- developer.openstack.org
static01-devstack-org:
- devstack.org
- www.devstack.org
static01-docs-opendev-org:
- docs.opendev.org
static01-docs-openstack-org:
- docs.openstack.org
static01-docs-starlingx-io:
- docs.starlingx.io
static01-eavesdrop-openstack-org:
- eavesdrop.openstack.org
static01-glance-openstack-org:
- glance.openstack.org
static01-git-airshipit-org:
- git.airshipit.org
static01-git-openstack-org:
- git.openstack.org
static01-git-starlingx-io:
- git.starlingx.io
static01-git-zuul-ci-org:
- git.zuul-ci.org
static01-governance-openstack-org:
- governance.openstack.org
static01-horizon-openstack-org:
- horizon.openstack.org
static01-keystone-openstack-org:
- keystone.openstack.org
static01-nova-openstack-org:
- nova.openstack.org
static01-meetings-opendev-org:
- meetings.opendev.org
static01-planet-openstack-org:
- planet.openstack.org
static01-service-types-openstack-org:
- service-types.openstack.org
static01-security-openstack-org:
- security.openstack.org
static01-specs-openstack-org:
- specs.openstack.org
static01-summit-openstack-org:
- summit.openstack.org
static01-swift-openstack-org:
- swift.openstack.org
static01-releases-openstack-org:
- releases.openstack.org
static01-tarballs-opendev-org:
- tarballs.opendev.org
static01-tarballs-openstack-org:
- tarballs.openstack.org
static01-zuul-ci-org:
- zuul-ci.org
- www.zuul-ci.org
- zuulci.org
- www.zuulci.org
static01-gating-dev:
- gating.dev
- www.gating.dev

@ -46,100 +46,100 @@
include_tasks: roles/letsencrypt-create-certs/handlers/restart_apache.yaml
# Static
- name: letsencrypt updated static01-opendev-org-main
- name: letsencrypt updated static-opendev-org-main
include_tasks: roles/letsencrypt-create-certs/handlers/restart_apache.yaml
- name: letsencrypt updated static01-ask-openstack-org
- name: letsencrypt updated static-ask-openstack-org
include_tasks: roles/letsencrypt-create-certs/handlers/restart_apache.yaml
- name: letsencrypt updated static01-docs-airshipit-org
- name: letsencrypt updated static-docs-airshipit-org
include_tasks: roles/letsencrypt-create-certs/handlers/restart_apache.yaml
- name: letsencrypt updated static01-ci-openstack-org
- name: letsencrypt updated static-ci-openstack-org
include_tasks: roles/letsencrypt-create-certs/handlers/restart_apache.yaml
- name: letsencrypt updated static01-cinder-openstack-org
- name: letsencrypt updated static-cinder-openstack-org
include_tasks: roles/letsencrypt-create-certs/handlers/restart_apache.yaml
- name: letsencrypt updated static01-developer-openstack-org
- name: letsencrypt updated static-developer-openstack-org
include_tasks: roles/letsencrypt-create-certs/handlers/restart_apache.yaml
- name: letsencrypt updated static01-devstack-org
- name: letsencrypt updated static-devstack-org
include_tasks: roles/letsencrypt-create-certs/handlers/restart_apache.yaml
- name: letsencrypt updated static01-docs-opendev-org
- name: letsencrypt updated static-docs-opendev-org
include_tasks: roles/letsencrypt-create-certs/handlers/restart_apache.yaml
- name: letsencrypt updated static01-docs-openstack-org
- name: letsencrypt updated static-docs-openstack-org
include_tasks: roles/letsencrypt-create-certs/handlers/restart_apache.yaml
- name: letsencrypt updated static01-docs-starlingx-io
- name: letsencrypt updated static-docs-starlingx-io
include_tasks: roles/letsencrypt-create-certs/handlers/restart_apache.yaml
- name: letsencrypt updated static01-eavesdrop-openstack-org
- name: letsencrypt updated static-eavesdrop-openstack-org
include_tasks: roles/letsencrypt-create-certs/handlers/restart_apache.yaml
- name: letsencrypt updated static01-glance-openstack-org
- name: letsencrypt updated static-glance-openstack-org
include_tasks: roles/letsencrypt-create-certs/handlers/restart_apache.yaml
- name: letsencrypt updated static01-git-airshipit-org
- name: letsencrypt updated static-git-airshipit-org
include_tasks: roles/letsencrypt-create-certs/handlers/restart_apache.yaml
- name: letsencrypt updated static01-git-starlingx-io
- name: letsencrypt updated static-git-starlingx-io
include_tasks: roles/letsencrypt-create-certs/handlers/restart_apache.yaml
- name: letsencrypt updated static01-git-openstack-org
- name: letsencrypt updated static-git-openstack-org
include_tasks: roles/letsencrypt-create-certs/handlers/restart_apache.yaml
- name: letsencrypt updated static01-git-zuul-ci-org
- name: letsencrypt updated static-git-zuul-ci-org
include_tasks: roles/letsencrypt-create-certs/handlers/restart_apache.yaml
- name: letsencrypt updated static01-governance-openstack-org
- name: letsencrypt updated static-governance-openstack-org
include_tasks: roles/letsencrypt-create-certs/handlers/restart_apache.yaml
- name: letsencrypt updated static01-horizon-openstack-org
- name: letsencrypt updated static-horizon-openstack-org
include_tasks: roles/letsencrypt-create-certs/handlers/restart_apache.yaml
- name: letsencrypt updated static01-keystone-openstack-org
- name: letsencrypt updated static-keystone-openstack-org
include_tasks: roles/letsencrypt-create-certs/handlers/restart_apache.yaml
- name: letsencrypt updated static01-meetings-opendev-org
- name: letsencrypt updated static-meetings-opendev-org
include_tasks: roles/letsencrypt-create-certs/handlers/restart_apache.yaml
- name: letsencrypt updated static01-nova-openstack-org
- name: letsencrypt updated static-nova-openstack-org
include_tasks: roles/letsencrypt-create-certs/handlers/restart_apache.yaml
- name: letsencrypt updated static01-planet-openstack-org
- name: letsencrypt updated static-planet-openstack-org
include_tasks: roles/letsencrypt-create-certs/handlers/restart_apache.yaml
- name: letsencrypt updated static01-service-types-openstack-org
- name: letsencrypt updated static-service-types-openstack-org
include_tasks: roles/letsencrypt-create-certs/handlers/restart_apache.yaml
- name: letsencrypt updated static01-specs-openstack-org
- name: letsencrypt updated static-specs-openstack-org
include_tasks: roles/letsencrypt-create-certs/handlers/restart_apache.yaml
- name: letsencrypt updated static01-security-openstack-org
- name: letsencrypt updated static-security-openstack-org
include_tasks: roles/letsencrypt-create-certs/handlers/restart_apache.yaml
- name: letsencrypt updated static01-summit-openstack-org
- name: letsencrypt updated static-summit-openstack-org
include_tasks: roles/letsencrypt-create-certs/handlers/restart_apache.yaml
- name: letsencrypt updated static01-swift-openstack-org
- name: letsencrypt updated static-swift-openstack-org
include_tasks: roles/letsencrypt-create-certs/handlers/restart_apache.yaml
- name: letsencrypt updated static01-releases-openstack-org
- name: letsencrypt updated static-releases-openstack-org
include_tasks: roles/letsencrypt-create-certs/handlers/restart_apache.yaml
- name: letsencrypt updated static01-tarballs-opendev-org
- name: letsencrypt updated static-tarballs-opendev-org
include_tasks: roles/letsencrypt-create-certs/handlers/restart_apache.yaml
- name: letsencrypt updated static01-tarballs-openstack-org
- name: letsencrypt updated static-tarballs-openstack-org
include_tasks: roles/letsencrypt-create-certs/handlers/restart_apache.yaml
- name: letsencrypt updated static01-zuul-ci-org
- name: letsencrypt updated static-zuul-ci-org
include_tasks: roles/letsencrypt-create-certs/handlers/restart_apache.yaml
- name: letsencrypt updated static01-gating-dev
- name: letsencrypt updated static-gating-dev
include_tasks: roles/letsencrypt-create-certs/handlers/restart_apache.yaml
# Grafana

@ -14,7 +14,7 @@
import pytest
testinfra_hosts = ['static01.opendev.org']
testinfra_hosts = ['static99.opendev.org']
def test_apache(host):
@ -30,6 +30,7 @@ def test_zuul_user(host):
assert authorized_keys.exists
static_names = (
'static99.opendev.org',
'static.opendev.org',
'static.openstack.org',
'files.openstack.org',
@ -37,37 +38,32 @@ static_names = (
@pytest.mark.parametrize("name", static_names)
def test_static_opendev_org(host, name):
cmd = host.run('curl --insecure '
'--resolve %s:443:127.0.0.1 https://%s/' %
cmd = host.run('curl --resolve %s:443:127.0.0.1 https://%s/' %
(name, name))
assert 'Index of /' in cmd.stdout
def test_ask_openstack_org(host):
cmd = host.run('curl --insecure '
'--resolve ask.openstack.org:443:127.0.0.1 '
cmd = host.run('curl --resolve ask.openstack.org:443:127.0.0.1 '
'https://ask.openstack.org/')
assert 'ask.openstack.org' in cmd.stdout
def test_docs_airshipit_org(host):
cmd = host.run('curl --insecure '
'--resolve docs.airshipit.org:443:127.0.0.1 '
cmd = host.run('curl --resolve docs.airshipit.org:443:127.0.0.1 '
'https://docs.airshipit.org/')
assert 'Airship documentation' in cmd.stdout
def test_governance_openstack_org(host):
cmd = host.run('curl --insecure '
'--resolve governance.openstack.org:443:127.0.0.1 '
cmd = host.run('curl --resolve governance.openstack.org:443:127.0.0.1 '
'https://governance.openstack.org/')
assert 'OpenStack Governance' in cmd.stdout
def test_security_openstack_org(host):
cmd = host.run('curl --insecure '
'--resolve security.openstack.org:443:127.0.0.1 '
cmd = host.run('curl --resolve security.openstack.org:443:127.0.0.1 '
'https://security.openstack.org/')
assert 'OpenStack Security Project' in cmd.stdout
def test_tarballs_openstack_org(host):
cmd = host.run('curl --insecure '
cmd = host.run('curl '
'--resolve tarballs.openstack.org:443:127.0.0.1 '
'--resolve tarballs.opendev.org:443:127.0.0.1 '
'https://tarballs.openstack.org/nova/')
@ -76,60 +72,51 @@ def test_tarballs_openstack_org(host):
assert 'https://tarballs.opendev.org/openstack/nova/' in cmd.stdout
def test_tarballs_opendev_org(host):
cmd = host.run('curl --insecure '
'--resolve tarballs.opendev.org:443:127.0.0.1 '
cmd = host.run('curl --resolve tarballs.opendev.org:443:127.0.0.1 '
'https://tarballs.opendev.org/openstack/nova/')
# An old file that should be present
assert 'nova-12.0.0.tar.gz' in cmd.stdout
def test_tarballs_opendev_org_redirects(host):
cmd = host.run('curl --insecure '
'--resolve tarballs.opendev.org:443:127.0.0.1 '
cmd = host.run('curl --resolve tarballs.opendev.org:443:127.0.0.1 '
'https://tarballs.opendev.org/openstack/afsmon/')
# Should be redirected to opendev/afsmon
assert '301 Moved Permanently' in cmd.stdout
assert 'https://tarballs.opendev.org/opendev/afsmon' in cmd.stdout
def test_specs_openstack_org(host):
cmd = host.run('curl --insecure '
'--resolve specs.openstack.org:443:127.0.0.1 '
cmd = host.run('curl --resolve specs.openstack.org:443:127.0.0.1 '
'https://specs.openstack.org/specs.opml')
assert 'OpenStack Specs Feeds' in cmd.stdout
def test_service_types_openstack_org(host):
cmd = host.run('curl --insecure '
'--resolve service-types.openstack.org:443:127.0.0.1 '
cmd = host.run('curl --resolve service-types.openstack.org:443:127.0.0.1 '
'https://service-types.openstack.org')
assert 'OpenStack Service Types Authority Data' in cmd.stdout
def test_releases_openstack_org(host):
cmd = host.run('curl --insecure '
'--resolve releases.openstack.org:443:127.0.0.1 '
cmd = host.run('curl --resolve releases.openstack.org:443:127.0.0.1 '
'https://releases.openstack.org')
assert 'OpenStack Releases: OpenStack Releases' in cmd.stdout
def test_developer_openstack_org(host):
cmd = host.run('curl --insecure '
'--resolve developer.openstack.org:443:127.0.0.1 '
cmd = host.run('curl --resolve developer.openstack.org:443:127.0.0.1 '
'https://developer.openstack.org')
assert 'OpenStack Docs: Application Development' in cmd.stdout
def test_docs_openstack_org(host):
cmd = host.run('curl --insecure '
'--resolve docs.openstack.org:443:127.0.0.1 '
cmd = host.run('curl --resolve docs.openstack.org:443:127.0.0.1 '
'https://docs.openstack.org')
# links to the latest, make sure it redirected us
assert '301 Moved Permanently' in cmd.stdout
def test_docs_opendev_org(host):
cmd = host.run('curl --insecure '
'--resolve docs.opendev.org:443:127.0.0.1 '
cmd = host.run('curl --resolve docs.opendev.org:443:127.0.0.1 '
'https://docs.opendev.org')
assert 'Index of /' in cmd.stdout
def test_docs_starlingx_io(host):
cmd = host.run('curl --insecure '
'--resolve docs.starlingx.io:443:127.0.0.1 '
cmd = host.run('curl --resolve docs.starlingx.io:443:127.0.0.1 '
'https://docs.starlingx.io')
# links to the latest, make sure it redirected us
assert 'Welcome to the StarlingX Documentation' \
@ -145,8 +132,7 @@ zuul_names = (
@pytest.mark.parametrize("name", zuul_names)
def test_zuulci_org(host, name):
cmd = host.run('curl --insecure '
'--resolve %s:443:127.0.0.1 https://%s/ ' %
cmd = host.run('curl --resolve %s:443:127.0.0.1 https://%s/ ' %
(name, name))
assert 'Zuul is an open source CI tool' in cmd.stdout
@ -173,7 +159,7 @@ def test_git_redirects(host, url, target):
assert '302 Found' in cmd.stdout
assert target in cmd.stdout
cmd = host.run('curl --insecure --resolve %s:443:127.0.0.1 https://%s' %
cmd = host.run('curl --resolve %s:443:127.0.0.1 https://%s' %
(hostname, url))
assert '302 Found' in cmd.stdout
assert target in cmd.stdout
@ -196,7 +182,7 @@ def test_doc_redirects(host, hostname, target):
assert '301 Moved Permanently' in cmd.stdout
assert target in cmd.stdout
cmd = host.run('curl --insecure --resolve %s:443:127.0.0.1 https://%s' %
cmd = host.run('curl --resolve %s:443:127.0.0.1 https://%s' %
(hostname, hostname))
assert '301 Moved Permanently' in cmd.stdout
assert target in cmd.stdout
@ -207,28 +193,24 @@ def test_summit_openstack_org(host):
assert '301 Moved Permanently' in cmd.stdout
assert 'https://openstack.org/summit/' in cmd.stdout
cmd = host.run('curl --insecure '
' --resolve summit.openstack.org:443:127.0.0.1'
cmd = host.run('curl --resolve summit.openstack.org:443:127.0.0.1'
' https://summit.openstack.org')
assert '301 Moved Permanently' in cmd.stdout
assert 'https://openstack.org/summit/' in cmd.stdout
def test_planet_openstack_org_redirects(host):
cmd = host.run('curl --insecure '
'--resolve planet.openstack.org:443:127.0.0.1 '
cmd = host.run('curl --resolve planet.openstack.org:443:127.0.0.1 '
'https://planet.openstack.org/')
assert '301 Moved Permanently' in cmd.stdout
assert 'https://opendev.org/openstack/openstack-planet' in cmd.stdout
def test_meetings_opendev_org(host):
cmd = host.run('curl --insecure '
'--resolve meetings.opendev.org:443:127.0.0.1 '
cmd = host.run('curl --resolve meetings.opendev.org:443:127.0.0.1 '
'https://meetings.opendev.org/')
assert 'IRC channels and meetings' in cmd.stdout
def test_eavesdrop_openstack_org(host):
cmd = host.run('curl --insecure '
'--resolve eavesdrop.openstack.org:443:127.0.0.1 '
cmd = host.run('curl --resolve eavesdrop.openstack.org:443:127.0.0.1 '
'https://eavesdrop.openstack.org/')
assert '301 Moved Permanently' in cmd.stdout
assert 'https://meetings.opendev.org' in cmd.stdout
@ -250,8 +232,7 @@ def test_ci_openstack_org(host, path, target):
assert '301 Moved Permanently' in cmd.stdout
assert target in cmd.stdout
cmd = host.run('curl --insecure '
' --resolve ci.openstack.org:443:127.0.0.1'
cmd = host.run('curl --resolve ci.openstack.org:443:127.0.0.1'
' https://ci.openstack.org%s' % path)
assert '301 Moved Permanently' in cmd.stdout
assert target in cmd.stdout

@ -297,7 +297,6 @@
files:
- inventory/base
- playbooks/service-static.yaml
- inventory/service/host_vars/static01.opendev.org.yaml
- inventory/service/group_vars/static.yaml
- playbooks/roles/iptables/
- playbooks/roles/static/

@ -1016,7 +1016,7 @@
nodes:
- name: bridge.openstack.org
label: ubuntu-bionic
- name: static01.opendev.org
- name: static99.opendev.org
label: ubuntu-bionic
vars:
run_playbooks:
@ -1030,7 +1030,7 @@
- playbooks/service-static.yaml
- testinfra/test_static.py
host-vars:
static01.opendev.org:
static99.opendev.org:
host_copy_output:
'/var/log/acme.sh/': logs
'/etc/apache2/': logs

Loading…
Cancel
Save