The with_ directives are discouraged now in place of use of loop: and/or
lookup filters. In the case of with_first_found, it confuses people
because with_ directives are usually a loop, but in this case it's
the task is always executed once. Using the first_found filter makes it
clearer that this is occuring.
While we're in there, remove uses of 'static: no'. Since 2.0 includes
are dynamic by default, so these are not necessary.
Change-Id: Ie429d7614b2f3322a646f46a8117d4b6ae29f737
The list of allowed hosts is comma separated, not colon separated.
Set exclusive: yes to ensure this is the *only* authorized key.
The zuul-executor group is the group for ze hosts. It's not a second
zuul-scheduler group.
Change-Id: I214482ce8931e697ada497048fcf12fa492b98b7
The purpose of the playbook is to update the system-config checkout, as
well as installing puppet modules and ansible roles.
Rename it, so that it's clearer what it does. Also, clean it up a bit.
We've gotten better at playbooks since we originally wrote this.
Change-Id: I793914ca3fc7f89cf019cf4cdf52acb7e0c93e60
There is a shared caching infrastructure in ansible now for inventory
and fact plugins. It needs to be configured so that our inventory access
isn't slow as dirt.
Unfortunately the copy of openstack.py in 2.6 is busted WRT to caching
because the internal API changed ... and we didn't have any test jobs
set up for it. This also includes a fixed copy of the plugin and
installs it into the a plugin dir.
Change-Id: Ie92e5d7eac4b7e4060a4e07cb29c5a6f2a16ae18
We put in IP restrictions on logging in as root on our servers. Add
bridge.openstack.org's IPs so that we can ansible from it.
Change-Id: Id1cd81c41806cd028d834fb56e1686687d3fb65d
We want to launch a new bastion host to run ansible on. Because we're
working on the transition to ansible, it seems like being able to do
that without needing puppet would be nice. This gets user management,
base repo setup and whatnot installed. It doesn't remove them from the
existing puppet, nor does it change the way we're calling anything that
currently exists.
Add bridge.openstack.org to the disabled group so that we don't try to
run puppet on it.
Change-Id: I3165423753009c639d9d2e2ed7d9adbe70360932
Remove some old ones which were in the wrong place and out of date.
Change-Id: I4303e66edc7d3dc00c455a0990b0b3be0f5f91a6
Depends-On: https://review.openstack.org/586699
We need to expand-contract our keypairs. This is the first of three
patches. The next will use this new keypair from nodepool. Then we can
remove the old one.
The new keypair object updates the ssh key for Shrews and removes
inactive old rooters.
Change-Id: I610e51b58a8b69c8d70c8be260e3a91e86247389
Packet Host and Platform 9 have generously agreed to donate some
compute resources to our testing efforts. Add Nodepool and
Puppetmaster credentials for them.
Change-Id: I705c4204abca060c35a1a417791a67229b78cd02
If a host is a member of the 'futureparser' group, pass the
'futureparser' option to the puppet role, which will turn on parser =
future in puppet.conf when manage_config is true and when the node isn't
already using puppet 4. Nodes can be added one at a time by adding them
to modules/openstack_project/files/puppetmaster/groups.txt.
Depends-On: https://review.openstack.org/572856
Change-Id: I54e19ef6164658da8e0e5bff72a1964b88b81242
Add a playbook to rerun install_puppet.sh with PUPPET_VERSION=4. Also
make the install_modules.sh script smarter about figuring out the puppet
version so that the update_puppet.yaml playbook, which updates the
puppet config and puppet modules but not the puppet package, does not
need to be changed.
When we're ready to start upgrading nodes, we'll add them to the puppet4
group in `modules/openstack_project/files/puppetmaster/groups.txt`.
Change-Id: Ic41d277b2d70e7c25669e0c07e668fb9479b8abf
Because we changed out the hostname of review.o.o for review01.o.o our
current playbooks will be broken. To fix this moving forward, we can
just switch to the group 'review' which includes the review01.o.o
host.
Change-Id: I149eacbc759f95087f2b0a0e44fcf0b49cae7ad6
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
When running the playbook, it's not immediately clear which task is
running without names. Add names. Also, update the whitespace to be more
in-line with how we write playbooks for zuul.
Change-Id: Ia189b8da6ded882aeb1fcff4932a1f9586027f80
We longer have any jobs or need to manage VMs in
tripleo-test-cloud-rh(1|2). This hardware still eventually be removed
so lets also remove it from our configuration.
Change-Id: I588ae945df15beceaf7a60bf6a65b1615b2074f0
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
We have puppet configured to write reports when it runs. We used to
collect these and inject them into puppetdb. Since we don't do this
anymore, they're just a giant pile of files we never see.
Enable managing the puppet.conf file from ansible and then also turn off
the reports.
Change-Id: I55bef052bddc9b9ff5de76a4f0b2ec07f93f158c
Following on from I166d9f669ea88663d4ffe70e25a6e908d11cf35f, add to
the cloud launcher. For now just add keys and security (no special
network setup).
Add a default image to the control plane account, as the cloud
currently doesn't have a xenial-based image. It needs a few special
properties to boot.
Change-Id: I846632219cbeb1f56eb0648861db0bfea3de7c3b
Now that zuulv3.openstack.org has been replaced by the larger
zuul01.openstack.org server, the former can be cleaned out of our
configuration in preparation for server deletion.
Change-Id: Icc1d545906e5615e2a205b98f364a084e1d22895
Since Ansible host inventory globs match against both host names and
host groups, use the zuul-scheduler group when referring to
zuul01.openstack.org and similarly-named hosts so as to avoid
inadvertently matching all members of the "zuul" host group with
zuul* (which includes the executors and mergers). Continue to match
zuulv3.openstack.org separately for now as it's not in the
zuul-scheduler group (and soon to be deleted anyway).
Change-Id: I3127d121ea344e1eb37c700d37f873e34edbb86e
To avoid the need for regular expression matching, switch to a
simple glob of zuul* covering zuulv3 and zuul01 servers. Now that
zuul-dev and zuulv3-dev are gone, this glob will only match the two
remaining hosts mentioned.
Change-Id: I2749ffa6c0e4d2ea6626d1ebde1d7b3ab49378bb
In preparation for replacing the zuulv3.openstack.org host with a
larger instance, set up the necessary support in
Puppet/Hiera/Ansible. While we're here, remove or replace old
references to the since-deleted zuul.openstack.org instance, and
where possible update documentation and configuration to refer to
the new zuul.openstack.org CNAME instead of the zuulv3.openstack.org
FQDN so as to smooth the future transition.
Change-Id: Ie51e133afb238dcfdbeff09747cbd2e53093ef84
We don't need a clean workspaces playbook, nor do we need to do anything
with that during renames. We don't need to reference machines that don't
exist in ansible groups. The launcher ssh config is not used by
anything.
Change-Id: Id3e9cddb06b6e47b6f07d9a39086f3b054b46bde
With the migration to zuulv3, there is no more zuul-launcher. This has
become zuul-executor, which has been moved into production.
Servers have already been deleted, lets also remove it from puppet.
Change-Id: Id2b53decdc63712460049f5fa9ed751e049d17ff
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
The set_hostname playbook, used by the launch-node script, needs
facts to determine which package manager it should use to uninstall
cloud-init. Remove the line which disabled fact gathering so that we
can build servers again.
Change-Id: Ic971d456f6d04273c9b981518614130e9b1c5898
This removes remaining references to internap (renamed to inap).
It also updates some items (cacti/nodepool logging) that were missed
in the rename.
Change-Id: Ibafd416e9e55aa458a50eb71922065a35e3d99f4
Bump ansible-playbook runs to 10% of our compute nodes, this is ~12
nodes at a time. We also max failures out to 100% because we actually
want to run puppet across all nodes, regardless of what fails.
Change-Id: I74b294820d8cd342fd7e5466ee63f198177412b4
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
We are having bandwidth issues in infracloud, lets experiment with
serial 1. We can adjust upwards if needed.
Change-Id: I89f0a1b197354e2d25d4f17ba29dd3da7d6586d4
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
In order to provide increased proxy cache capacity, increase the
mirror flavor's disk size in Infra-cloud to 250GiB. Other providers
will get Cinder volumes added as needed.
Change-Id: I56130167e94237b93b3bdbfd1334eb97c76836fa
This should give us connectivity to the outside world with NAT'd
internal IP addressing.
Note that we can't add the router to the template because the external
network name will be different across clouds and we have to pass in the
subnet lists which may vary as well.
Change-Id: Iea225c71d0d8e644cbaf709554d02d130ad21c18
Currently puppet fails to run on our baremetal servers for infracloud.
While this is an issue, it should not block puppet from running on our
controller or compute nodes.
Change-Id: I190af6cfc63006cb03686cd501998e4e06d350b1
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
We need to ensure ovh is properly setup with our SSH keypairs for
nodepool.
Change-Id: I2a02dfb5da2ac0af087d502ae8143047e3d1b12c
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
Copy our current infra-root list from user.pp into cloud_layouts.yml.
Change-Id: Ic339f6879782a9f9d7d92a445160c5b0949a698b
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
Because rackspace doesn't support security groups, we need to create
openstackci-keypairs.
Change-Id: I549c5e99554eb876b872a08989dc0345a799ff00
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
Since we are moving forward with removing our baked in SSH keys for
our images, we now need to move our public keys into our clouds. This
will allow nodepool to inject them into metadata for glean.
Change-Id: I0ff9db47a0845ed9d038792383624af4bd34d525
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
We are in the process of shutting down puppetdb.o.o, so stop pushing
reports to it.
Change-Id: Ib27b21c3fb2cd149e57432fd511129a5c8ecc3e9
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
This fixes the issues we have with our rename_repos.yaml file. We are
also skipping additional failures for now, which will be cleaned up in
a follow up patch.
Change-Id: I726535e195a292e3f2d457f0ed039d01bb96c66b
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
Currently, if review.o.o takes more then 30mins to run puppet, it will
be aborted. Up this to 60m.
Change-Id: I98e384544d5104572ad252b5dab88e06762b87a9
Depends-On: Id42ba80a5118a9f93e45619ac6ecc5baa774549a
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
When I919ba42b0d22126719daa7ad308f75ce021720b7 merged, it introduced
a few regressions into our process:
* Github renaming/transferring was dropped
* Switched to a very slow (for our environment) Zuul stopping
method
* it advocated for composing a rename parameters file very late
in the process
This change fixes the above issues. It also updates the
documentation to note that Puppet should be stopped well in advance
of the maintenance window, and updates the playbook to no longer run
an offline Gerrit reindex (since online reindexing is now
supported).
Change-Id: Ie249214c0d1b1df6c66d4910002e35d8c17c3b69
In the infracloud, the Member role is not created by default.
We created that with a previous change by adding it to the launcher.
Now we associate that role to the openstackci/openstackzuul user/projects,
so those users are members of their corresponding projects.
Change-Id: I9147b253c7f747f435c773932dc4a8aad1189799
We need to create these roles, so we can associate users with projects.
Change-Id: I29af32c9b0f99c584b6ed76b346b1b117d05b277
Depends-On: I2df8503bb713827f0f04691c2f259dc9541c9c83
The servers are still currently created by launch-node, I'll revert
this commit when I put the pre/post create/delete actions per resource
on the launcher role.
Change-Id: I0a6401c9d783b9c3876ebb1f9c8b144f75d7abb2
It was discussed with other members of the Infra team that this
file would be better place on the playbooks folder, since the
run_launcher is located there.
Change-Id: I752ee592d3ffd8be4fd4ad29dbf73df443f28674
Now that we've confirmed ansible-playbook works as expected, lets
enable the free strategy by default.
While playbooks with singles hosts will not benefit from this, we add
it to be consistent with our playbooks.
Change-Id: Ia6abdfaf5c122f88ead2272c8700e2c1f33c5449
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
In an effort to improve performance, switch out strategy[1] to free.
This will allow each ansible host to run until the end of the play as
fast as it can.
[1] http://docs.ansible.com/ansible/playbooks_strategies.html
Change-Id: I86588154b71e69399be930fc78be7c17f54fd9dd
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
Running this playbook on the puppetmaster we consistently run into ssh
failures due to async reconnecting periodically and network issues
between hosts. We can address this by starting a single connection
without async and polling on that which appears to be the default
wait_for behavior. Testing of this seems to indicate it is more
reliable.
Change-Id: Iec72e2c0d099c0e28bc4b4b48608a03b3e66b4c0
Add support so we can run the playbook as non-root user.
Change-Id: I05af471417ba58a985c24dc0ea2c43f1c7e24a4b
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
We nolonger need it as we don't have jenkins masters any more.
Change-Id: I8117a6f4afb9f65a1400fad090594efd260c3bec
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
We'll wait up to 3hr 10mins for zuul-launchers to shutdown.
Change-Id: I880748704b6cae5a25c21326d6374ac71f4c9e1a
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
This is the runner for the ansible cloud launcher role.
Change-Id: Iad9ce14905e89cb875c0cf92dfd8093c3a8d4e1c
Depends-On: Ia775598090471b80be75624a6a6a0649622799e8
We're already on the host, and this defaults to localhost, so this
is simpler and doesn't go through the apache proxy.
Change-Id: Iac1047dc0a482d21466ab062f3aa3b0ef9144f38
Running puppet remotely in an ad-hoc manner on disabled hosts is mildly
complex. To facilitate, have a wide open playbook that we always run
with --limit - and a shell script to help us type less.
Change-Id: I629072dcada38d0465d351b1b99828466405372f
It's fine right now with 5, but over time if we keep a flat namespae,
which is not necessary, it's just going to get ugly.
Change-Id: I07a143f45f2eb100c231ea1b7dd617b40f8f231c
We are only deploying West for now, so just doing West.
When we get East in production, we would update this playbook.
Unfortunate there is no Ansible module or Puppet resources to set
quotas per-project, thus using regular shell module in Ansible.
Change-Id: Ib884508bebedc9f88fac242711af98fc0c4d95ec
Turns out we have had many issues with random servers having
wrong hostname and /etc/hosts info.
This playbook/role allows to configure that by passing
-e "target=<hostname>" as ansible-playbook parameter.
Change-Id: I73939ebc65211a840bb41370c22b111112389716
In a clean deploy of infra cloud, the puppet environment
is not configured from scratch. That will prevent puppet to run
because it won't find the /opt/system-config/production/modules.
The config option of the ansible role will configure properly
all settings before trying to apply it, and things will work
properly.
Change-Id: I736e10623fb3ba90b3320cc20758a18c70930be0
Depends-On: I6cb8dff569f2cca8bca7359412d01cc7ec009c54
Without this patch, we would run infracloud in its playbook, then again
in the 'everybody else' playbook.
Co-Authored-By: Colleen Murphy <colleen@gazlene.net>
Change-Id: I3de1de8f0f74e52a443c0b7a6ef6ae0a2cf7e801
Add separate playbook for infacloud nodes to ensure they run in the
correct order - baremetal -> controller -> compute.
Baremetal is intentionally left out, it is not ready yet.
All 'disabled' flags on infracloud hosts are turned off. This patch
landing turns on management of the infracloud.
Co-Authored-By: Yolanda Robla <info@ysoft.biz>
Co-Authored-By: Spencer Krum <nibz@spencerkrum.com>
Change-Id: Ieeda072d45f7454d6412295c2c6a0cf7ce61d952
The puppet ansible module is growing a flag to be able to send stdout to
syslog. It's growing that because we want to use it. Let's.
Change-Id: I22b1d0e1fb635f2c626d75a11764725c8753bf24
At long last, the day of reckoning is here. Run puppet apply and then
copy the log files back and post them to puppetdb.
Change-Id: I919fea64df0fbb8681e91ac9425b4c43760bb3dd
We don't need to rsync to ourselves. Best case it's a no-op. Worst case
something weird happens and we overwrite ourselves while running.
Change-Id: I890ea487d7a6129b7477b6d17b6a7e3c1904bade
When we do it as a second playbook, the failure to copy updated code
cannot prevent puppet from running.
Change-Id: I94b06988a20da4c0c2cf492485997ec49c3dca13
Depends-On: I22b7a21778d514a0a1ab04a76f03fdc9c58a05b3
There are a few things that are run as part of run_all.sh that are
not logged into puppet_run_all.log - namely git cloning, module installation
and ansible role installation. Let's go ahead and do those in a playbook
so that we can see their output while we're watching the log file.
Change-Id: I6982452f1e572b7bc5a7b7d167c1ccc159c94e66
We're not ready to move from puppet inventory to openstack inventory
just yet, so don't actually swap the dynamic inventory plugin. But, add
it to the system so that running manual tests of all of the pieces is
possible.
Add the currently administratively disabled hosts to the disabled group
so that we can verify this works.
Change-Id: I73931332b2917b71a008f9213365f7594f69c41e
One step before flipping the switch, start copying hieradata, even
though we're still using agent, so that we can verify as much as we
want.
Change-Id: Iae63fd056cdb17aedd6526b9cbc1d83037ddcbb3
We use a symlink into /opt/system-config to make the hiera.yaml config
sane. Make sure it's there.
Change-Id: I5e9681ac8fca71ce2f439eed3ef1281ba228d5b2
If we're going to run puppet apply on all of our nodes, they need
the puppet modules installed on them first.
Change-Id: I84b80818fa54d1ddc4d46fead663ed4212bb6ff3
As we're using these roles, we'll want to pass potentially different
values to different of our hosts over time. For instance, we may want to
set the jenkins servers to start using puppet apply before we get all
the hosts there. Since we run most of the hosts in a big matching
mechanism, the way we can pass different input values to each host.
Change-Id: I5698355df0c13cd11fe5987787e65ee85a384256
/etc/ansible/playbooks isn't actually a thing, it was just a convenient
place to put things. However, to enable puppet apply, we're going to
want a group_vars directory adjacent to the playbooks, so having them be
a subdirectory of the puppet module and installed by it is just extra
complexity. Also, if we run out of system-config, then it'll be easier
to work with things like what we do with puppet environments for testing
things.
Change-Id: I947521a73051a44036e7f4c45ce74a79637f5a8b
Monty Taylor