Preparing to implement push mode


#1

I have a few tickets to do before I can start push mode, but I typed up some rough notes here:

The idea is that the opsmop file should be usable for both local and push mode depending on whether you use opsmop-push or opsmop.

A super-fast pull mode is still something I really want to do, right after this.

Because of architectural choices, push mode in OpsMop should also be lightning fast. (Just one rsync + just one SSH op).


#2

Today I've added many things that help get push mode closer to reality, with the expectation that it will actually be a thing sometime next week.

There's now a hidden option "--event-stream" to the local CLI, where the output is a bunch of JSON documents instead of something human friendly. When we do parallel SSH for push, this will be useful to parse all the output efficiently.

There's also a TOML inventory class , with my rough draft of what this would look like here:

Naturally inventory could come from anything, I believe the most popular dynamic inventory source would be from AWS tags, and that is probably going to be an early addition from someone

You can see how short the TOML implementation is here, to see how easy the AWS one would be:

Basically the system just needs to return some dictionaries in a particular format.

Ansible fans may ask "where are group_vars/ and host_vars/" ... I'm not opposed to seeing those, and they may come later also.

They may also say "where are nested groups!" ... and that's something that we can also add once the basics get working.

If you scroll down to the bottom of this file, you can see how I roughly expect to see tags and groups map together:

(see the commented out "inventory.map").

Also to to the top of the file you can see the same file can be used for both bin/opsmop (local) and bin/opsmop-push.

The next big effort is of course to actually implement the rest of it :)

For those who are wanting things other than SSH-push, I will be making this pluggable and also be adding a pull mode right after I get done with push to a level that I'm happy with it.

There will be a lot of room for playing with push performance and output to get it right, but the first step is to get it working.

So far, this is fairly clean, and I think we can avoid all the trouble with ansible inventory design in the past.


#3

I thought about this a bit and the inventory will work better if we declare it up top, and each Role can have a method like:

def push_inventory(self):
    return inventory.filter(groups=['webservers'])

This means the tag/mapping stuff can go away


#4

I've added a stub program in "bin/" for opsmop-push that does nothing just yet, but what it is going to do is walk the policies, walk the roles, and for each role push them to all the hosts.

Before doing that, it's going to walk all roles, get all hosts in any role, and then rsync the config directory to them.

As far as I can see this is only TWO ssh OPs total, and I plan to just shell out to rsync and then SSH and recommend people use ssh-agent to handle any keys that might be involved.

We're going to login as whoever and then run opsmop via sudo.

You will need to have opsmop installed but I'm probably going to make a provision to have a step where it can push something like a bootstrap.sh and run it, which could be used to install opsmop.

I'll probably also do something like "--limit-groups" and "--limit-hosts" that further constrains the host and group patterns in the configuration, once I get the basics in.