This removes remaining references to internap (renamed to inap).
It also updates some items (cacti/nodepool logging) that were missed
in the rename.
Change-Id: Ibafd416e9e55aa458a50eb71922065a35e3d99f4
Bump ansible-playbook runs to 10% of our compute nodes, this is ~12
nodes at a time. We also max failures out to 100% because we actually
want to run puppet across all nodes, regardless of what fails.
Change-Id: I74b294820d8cd342fd7e5466ee63f198177412b4
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
We are having bandwidth issues in infracloud, lets experiment with
serial 1. We can adjust upwards if needed.
Change-Id: I89f0a1b197354e2d25d4f17ba29dd3da7d6586d4
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
In order to provide increased proxy cache capacity, increase the
mirror flavor's disk size in Infra-cloud to 250GiB. Other providers
will get Cinder volumes added as needed.
Change-Id: I56130167e94237b93b3bdbfd1334eb97c76836fa
This should give us connectivity to the outside world with NAT'd
internal IP addressing.
Note that we can't add the router to the template because the external
network name will be different across clouds and we have to pass in the
subnet lists which may vary as well.
Change-Id: Iea225c71d0d8e644cbaf709554d02d130ad21c18
Currently puppet fails to run on our baremetal servers for infracloud.
While this is an issue, it should not block puppet from running on our
controller or compute nodes.
Change-Id: I190af6cfc63006cb03686cd501998e4e06d350b1
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
We need to ensure ovh is properly setup with our SSH keypairs for
nodepool.
Change-Id: I2a02dfb5da2ac0af087d502ae8143047e3d1b12c
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
Copy our current infra-root list from user.pp into cloud_layouts.yml.
Change-Id: Ic339f6879782a9f9d7d92a445160c5b0949a698b
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
Because rackspace doesn't support security groups, we need to create
openstackci-keypairs.
Change-Id: I549c5e99554eb876b872a08989dc0345a799ff00
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
Since we are moving forward with removing our baked in SSH keys for
our images, we now need to move our public keys into our clouds. This
will allow nodepool to inject them into metadata for glean.
Change-Id: I0ff9db47a0845ed9d038792383624af4bd34d525
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
We are in the process of shutting down puppetdb.o.o, so stop pushing
reports to it.
Change-Id: Ib27b21c3fb2cd149e57432fd511129a5c8ecc3e9
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
This fixes the issues we have with our rename_repos.yaml file. We are
also skipping additional failures for now, which will be cleaned up in
a follow up patch.
Change-Id: I726535e195a292e3f2d457f0ed039d01bb96c66b
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
Currently, if review.o.o takes more then 30mins to run puppet, it will
be aborted. Up this to 60m.
Change-Id: I98e384544d5104572ad252b5dab88e06762b87a9
Depends-On: Id42ba80a5118a9f93e45619ac6ecc5baa774549a
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
When I919ba42b0d22126719daa7ad308f75ce021720b7 merged, it introduced
a few regressions into our process:
* Github renaming/transferring was dropped
* Switched to a very slow (for our environment) Zuul stopping
method
* it advocated for composing a rename parameters file very late
in the process
This change fixes the above issues. It also updates the
documentation to note that Puppet should be stopped well in advance
of the maintenance window, and updates the playbook to no longer run
an offline Gerrit reindex (since online reindexing is now
supported).
Change-Id: Ie249214c0d1b1df6c66d4910002e35d8c17c3b69
In the infracloud, the Member role is not created by default.
We created that with a previous change by adding it to the launcher.
Now we associate that role to the openstackci/openstackzuul user/projects,
so those users are members of their corresponding projects.
Change-Id: I9147b253c7f747f435c773932dc4a8aad1189799
We need to create these roles, so we can associate users with projects.
Change-Id: I29af32c9b0f99c584b6ed76b346b1b117d05b277
Depends-On: I2df8503bb713827f0f04691c2f259dc9541c9c83
The servers are still currently created by launch-node, I'll revert
this commit when I put the pre/post create/delete actions per resource
on the launcher role.
Change-Id: I0a6401c9d783b9c3876ebb1f9c8b144f75d7abb2
It was discussed with other members of the Infra team that this
file would be better place on the playbooks folder, since the
run_launcher is located there.
Change-Id: I752ee592d3ffd8be4fd4ad29dbf73df443f28674
Now that we've confirmed ansible-playbook works as expected, lets
enable the free strategy by default.
While playbooks with singles hosts will not benefit from this, we add
it to be consistent with our playbooks.
Change-Id: Ia6abdfaf5c122f88ead2272c8700e2c1f33c5449
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
In an effort to improve performance, switch out strategy[1] to free.
This will allow each ansible host to run until the end of the play as
fast as it can.
[1] http://docs.ansible.com/ansible/playbooks_strategies.html
Change-Id: I86588154b71e69399be930fc78be7c17f54fd9dd
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
Running this playbook on the puppetmaster we consistently run into ssh
failures due to async reconnecting periodically and network issues
between hosts. We can address this by starting a single connection
without async and polling on that which appears to be the default
wait_for behavior. Testing of this seems to indicate it is more
reliable.
Change-Id: Iec72e2c0d099c0e28bc4b4b48608a03b3e66b4c0
Add support so we can run the playbook as non-root user.
Change-Id: I05af471417ba58a985c24dc0ea2c43f1c7e24a4b
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
We nolonger need it as we don't have jenkins masters any more.
Change-Id: I8117a6f4afb9f65a1400fad090594efd260c3bec
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
We'll wait up to 3hr 10mins for zuul-launchers to shutdown.
Change-Id: I880748704b6cae5a25c21326d6374ac71f4c9e1a
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
This is the runner for the ansible cloud launcher role.
Change-Id: Iad9ce14905e89cb875c0cf92dfd8093c3a8d4e1c
Depends-On: Ia775598090471b80be75624a6a6a0649622799e8
We're already on the host, and this defaults to localhost, so this
is simpler and doesn't go through the apache proxy.
Change-Id: Iac1047dc0a482d21466ab062f3aa3b0ef9144f38
Running puppet remotely in an ad-hoc manner on disabled hosts is mildly
complex. To facilitate, have a wide open playbook that we always run
with --limit - and a shell script to help us type less.
Change-Id: I629072dcada38d0465d351b1b99828466405372f
It's fine right now with 5, but over time if we keep a flat namespae,
which is not necessary, it's just going to get ugly.
Change-Id: I07a143f45f2eb100c231ea1b7dd617b40f8f231c
We are only deploying West for now, so just doing West.
When we get East in production, we would update this playbook.
Unfortunate there is no Ansible module or Puppet resources to set
quotas per-project, thus using regular shell module in Ansible.
Change-Id: Ib884508bebedc9f88fac242711af98fc0c4d95ec
Turns out we have had many issues with random servers having
wrong hostname and /etc/hosts info.
This playbook/role allows to configure that by passing
-e "target=<hostname>" as ansible-playbook parameter.
Change-Id: I73939ebc65211a840bb41370c22b111112389716
In a clean deploy of infra cloud, the puppet environment
is not configured from scratch. That will prevent puppet to run
because it won't find the /opt/system-config/production/modules.
The config option of the ansible role will configure properly
all settings before trying to apply it, and things will work
properly.
Change-Id: I736e10623fb3ba90b3320cc20758a18c70930be0
Depends-On: I6cb8dff569f2cca8bca7359412d01cc7ec009c54
Without this patch, we would run infracloud in its playbook, then again
in the 'everybody else' playbook.
Co-Authored-By: Colleen Murphy <colleen@gazlene.net>
Change-Id: I3de1de8f0f74e52a443c0b7a6ef6ae0a2cf7e801
Add separate playbook for infacloud nodes to ensure they run in the
correct order - baremetal -> controller -> compute.
Baremetal is intentionally left out, it is not ready yet.
All 'disabled' flags on infracloud hosts are turned off. This patch
landing turns on management of the infracloud.
Co-Authored-By: Yolanda Robla <info@ysoft.biz>
Co-Authored-By: Spencer Krum <nibz@spencerkrum.com>
Change-Id: Ieeda072d45f7454d6412295c2c6a0cf7ce61d952
The puppet ansible module is growing a flag to be able to send stdout to
syslog. It's growing that because we want to use it. Let's.
Change-Id: I22b1d0e1fb635f2c626d75a11764725c8753bf24
At long last, the day of reckoning is here. Run puppet apply and then
copy the log files back and post them to puppetdb.
Change-Id: I919fea64df0fbb8681e91ac9425b4c43760bb3dd
We don't need to rsync to ourselves. Best case it's a no-op. Worst case
something weird happens and we overwrite ourselves while running.
Change-Id: I890ea487d7a6129b7477b6d17b6a7e3c1904bade
When we do it as a second playbook, the failure to copy updated code
cannot prevent puppet from running.
Change-Id: I94b06988a20da4c0c2cf492485997ec49c3dca13
Depends-On: I22b7a21778d514a0a1ab04a76f03fdc9c58a05b3
There are a few things that are run as part of run_all.sh that are
not logged into puppet_run_all.log - namely git cloning, module installation
and ansible role installation. Let's go ahead and do those in a playbook
so that we can see their output while we're watching the log file.
Change-Id: I6982452f1e572b7bc5a7b7d167c1ccc159c94e66
We're not ready to move from puppet inventory to openstack inventory
just yet, so don't actually swap the dynamic inventory plugin. But, add
it to the system so that running manual tests of all of the pieces is
possible.
Add the currently administratively disabled hosts to the disabled group
so that we can verify this works.
Change-Id: I73931332b2917b71a008f9213365f7594f69c41e
One step before flipping the switch, start copying hieradata, even
though we're still using agent, so that we can verify as much as we
want.
Change-Id: Iae63fd056cdb17aedd6526b9cbc1d83037ddcbb3
We use a symlink into /opt/system-config to make the hiera.yaml config
sane. Make sure it's there.
Change-Id: I5e9681ac8fca71ce2f439eed3ef1281ba228d5b2
If we're going to run puppet apply on all of our nodes, they need
the puppet modules installed on them first.
Change-Id: I84b80818fa54d1ddc4d46fead663ed4212bb6ff3
As we're using these roles, we'll want to pass potentially different
values to different of our hosts over time. For instance, we may want to
set the jenkins servers to start using puppet apply before we get all
the hosts there. Since we run most of the hosts in a big matching
mechanism, the way we can pass different input values to each host.
Change-Id: I5698355df0c13cd11fe5987787e65ee85a384256
/etc/ansible/playbooks isn't actually a thing, it was just a convenient
place to put things. However, to enable puppet apply, we're going to
want a group_vars directory adjacent to the playbooks, so having them be
a subdirectory of the puppet module and installed by it is just extra
complexity. Also, if we run out of system-config, then it'll be easier
to work with things like what we do with puppet environments for testing
things.
Change-Id: I947521a73051a44036e7f4c45ce74a79637f5a8b
James E. Blair