14:00:55 <mattmceuen> #startmeeting airship
14:00:58 <mattmceuen> #topic Rollcall
14:01:00 <openstack> Meeting started Tue Feb 26 14:00:55 2019 UTC and is due to finish in 60 minutes.  The chair is mattmceuen. Information about MeetBot at http://wiki.debian.org/MeetBot.
14:01:01 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
14:01:04 <mattmceuen> Hello everyone!
14:01:05 <openstack> The meeting name has been set to 'airship'
14:01:08 <dwalt> o/
14:01:13 <michael-beaver> o/
14:01:25 <mattmceuen> Here's our agenda for today - please take a moment to add anthing you'd like to discuss: https://etherpad.openstack.org/p/airship-meeting-2019-02-26
14:01:33 <georgk3> hi
14:01:51 <levmorgan> o/
14:01:55 <portdirect> ヅ
14:02:03 <arunkant> o/
14:02:08 <b-str> o/
14:02:18 <roman_g> o/
14:02:40 <mattmceuen> #topic Consider what onboarding environment seems the most valuable
14:02:59 <aaronsheffield> o/
14:03:02 <mattmceuen> So this one was sthussey's; unfortunately he's not in today
14:03:03 <pas-ha> o/
14:03:09 <mattmceuen> o/ everyone
14:03:44 <mattmceuen> sthussey had brought up a good point that we have a number of different environments that are recommended for onboarding / reference type things
14:04:05 <pas-ha> my question would also be 'is there a doc describing a dev process'
14:04:18 <pas-ha> like if I need to patch and test
14:04:24 <mattmceuen> Airskiff
14:04:25 <mattmceuen> Airship-in-a-bottle local VM
14:04:25 <mattmceuen> Airship-in-a-bottle multi-VM
14:04:25 <mattmceuen> Treasuremap - Seaworthy
14:05:07 <mattmceuen> There is a little bit of that in the Airskiff documentation, but not much to speak of pas-ha.  I think that's a very good value-add for the docs
14:05:08 <pas-ha> like a path from local repo -> patch -> build new container? -> push it to local docker? -> redeploy?
14:05:15 <mattmceuen> A "day in the life of a developer" kind of thing
14:05:30 <pas-ha> yes, smth like that
14:05:37 <georgk3> +1
14:05:42 <pas-ha> figuring that out myself for now
14:06:12 <mattmceuen> #action mattmceuen to create a storyboard item for documenting the Airship development lifecycle
14:07:06 <mattmceuen> My thought is that for the different "things" up above, they are each a combo of a couple different characteristics:
14:07:10 <mattmceuen> 1) a deployment approach
14:07:21 <mattmceuen> 2) a collection of manifests
14:07:49 <mattmceuen> Whatever else we do, I want to try to align our manifests to one another as much as possible
14:08:12 <mattmceuen> dwalt has a patchset out for example that takes the airskiff manifests and aligns them more closely to the treasuremap globals
14:09:26 <mattmceuen> I think maintenance of different, segmented manifests sets has been part of the maintenance challenge that sthussey wants to avoid
14:09:33 <dwalt> This might be a step back, but as far as lifecycle goes, I usually try to run the component locally before deploying any changes into one of our "development environments (i.e. Airskiff, AIAB)
14:09:53 <dwalt> For example, you can run Armada as a local Python project against KubeADM cluster, and it will find Tiller
14:10:19 <dwalt> Deckhand and Shipyard also support similar dev setups, but they are buried a bit in the docs
14:11:11 <mattmceuen> Promenade has a great "prom only" test environment, which spins up a 4-node VM cluster and beats it up.  This is what Airship-in-a-Bottle multinode (the tooling) was modeled after
14:11:29 <mattmceuen> Airship-in-a-Bottle (the manifest set) is something that could be aligned into the treasuremap globals
14:12:25 <mattmceuen> Let me ask for y'alls opinions -- what do you find valuable amongst these things?  I've heard the opinion that "developers don't tend to use airship single node", but that's the main onboarding path for new folks.   Would AiaB multinode serve that purpose, perhaps even better?
14:14:21 <mattmceuen> I for one use Airskiff and AiaB multinode for my own work, depending on what I'm working on
14:15:02 <roman_g> For professional development people do probably use big bare-metal servers with lots of RAM.
14:15:26 <roman_g> This is not something to expect to have from those who only want to have a look at Airship
14:15:30 <dwalt> For me, it depends on the component. I tend to use Airskiff (single node) for the software delivery components and AIAB multimode in other cases
14:15:58 <michael-beaver> I for one use the multinode a lot, but I think it is very prohibitive for someone without a dedicated lab
14:16:38 <mattmceuen> The nice thing about multinode is that it can deploy a variable number of nodes, based on configs... even perhaps a single node :)
14:17:19 <mattmceuen> So that's where my head is at with, we could do more things with less moving pieces if we aligned our manifests together and configured AiaB multinode scripts to be the single-node demo environment as well
14:17:42 <mattmceuen> Any disagreement with this general sentiment?
14:18:19 <levmorgan_> Sounds good to me.
14:18:27 <georgk3> I like the multinode environment a lot, so +1
14:18:37 <michael-beaver> Maybe this is trivial, but another problem with running it is you need a Host OS capable of nested-virt, so windows using virtualbox is a no go ATM, but overall I don't disagree with the sentiment that the multinode gate would help us have less moving pieces
14:19:35 <dwalt> ++ mattmceuen. It'd be a great step moving our development environments closer in-line with our "reference" architecture
14:20:09 <mattmceuen> Cool, let's circle back with sthussey when he's back
14:20:27 <mattmceuen> good point michael-beaver and we should keep that in mind
14:20:38 <mattmceuen> we've also been talking about aligning AiaB manifests to treasuremap for a long time
14:20:55 <mattmceuen> #action mattmceuen create a Storyboard item for aligning AiaB manifests to Treasuremap
14:21:13 <mattmceuen> anything else on this topic for now guys?
14:22:14 <mattmceuen> #topic Ironic driver spec
14:22:52 <mattmceuen> pas-ha and some other folks have been going some work on the spec for adding Ironic support to Drydock:
14:23:01 <mattmceuen> https://review.openstack.org/#/c/613358/
14:23:19 <mattmceuen> roman_g, you wanted to sync up on that here, right?
14:25:02 <roman_g> mattmceuen: not really to sync up, but may be we have Hemmanth here?
14:25:04 <roman_g> hemanth_n:
14:25:33 <roman_g> If not, let's move on.
14:26:22 <mattmceuen> Ok.  I need to catch up on the latest discussion in the PS as well; pas-ha & roman_g should we plan to touch on this next week?
14:26:40 <roman_g> Yes. Move to the next week.
14:26:51 <roman_g> Would also be good to discuss on design call
14:26:54 <roman_g> probably
14:26:57 <mattmceuen> If there's anything specific we need to grab hemanth_n's attention on we can mailing list it as well
14:27:06 <roman_g> True.
14:28:11 <mattmceuen> Next up:
14:28:15 <mattmceuen> #topic Airship talks accepted into the Denver Summit
14:28:28 <mattmceuen> So, we have some Airship talks accepted into the Denver Summit!
14:28:45 <mattmceuen> Unfortunately, the summit website appears to not be functioning today, so I can't tell you what they all are! :D
14:29:06 <mattmceuen> So many great Airship talks that the database crashed perhaps
14:29:17 <mattmceuen> will share next time
14:29:28 <mattmceuen> #topic Design meeting(s) for Docker, Kernel, OS, Security Patches, etc
14:30:18 <mattmceuen> Just wanted to make sure folks were aware that Rodolfo has scheduled a design call later today (and may end up being recurring as needed) to dive deep into the topic of how to best orchestrate heavyweight, low-level changes across Airship sites
14:30:40 <mattmceuen> Things like operating system upgrades, kubelet upgrades, docker, kernel, etc
14:30:49 <mattmceuen> As well as simple cluster-wide reboots when needed
14:31:13 <georgk3> interesting, thanks for the info
14:31:29 <roman_g> interesting. not in my calendar
14:31:32 <evgenyl> Where can I get an invite?
14:31:35 <roman_g> will ask. thank you
14:32:25 <mattmceuen> The intention has always been for Airship to be able to reprovision hosts in a way that preserves the local data for the running workloads (e.g. VMs)  . This is not quite finished yet, but will allow for e.g. host-by-host or failure-domain-by-domain reprovisionings of the cluster with e.g. a new patched kernel
14:32:42 <mattmceuen> However, that's also a heavyweight operation and there are lighterweight things we can do as well
14:32:49 <mattmceuen> Anyway - please join if you'd like to discuss more
14:33:07 <roman_g> Good to have it recorded and published
14:33:08 <mattmceuen> The invite should be out on the Mailing List, if you go back to the archives from yesterday or Friday I believe
14:33:10 <evgenyl> Oh, sorry, it was in the mailing list, I somehow missed it.
14:33:23 <mattmceuen> Don't worry evgenyl, I am good at missing all kinds of email
14:33:37 <mattmceuen> three and a half hours from now
14:33:53 <mattmceuen> Moving on:
14:33:57 <mattmceuen> #topic Divingbell gates are broken in upstream
14:34:05 <mattmceuen> roman_g this is yours, go for it
14:34:23 <roman_g> Pretty much everything is in the description https://etherpad.openstack.org/p/airship-meeting-2019-02-26
14:34:29 <roman_g> Update of Helm from version 2.11.0 to 2.12.1 and then to 2.12.3 broke host and label `overrides:` functionality for the Airship Divingbell.
14:34:37 <roman_g> This affects Divingbell development (voting gates not working), and partially other projects like openstack/openstack-helm-infra, which have Divingbell as a non-voting gate
14:34:43 <roman_g> Roman is looking for an assistance
14:35:23 <roman_g> Issue in one line: secret contents which is used for the host falling under override is identical to the secret used by default
14:35:39 <Nishant_> I see a topic dedicated to the overrides issue in openstack-helm meeting today starting 9 AM CST
14:35:47 <mattmceuen> awesome
14:36:19 <mattmceuen> Yes this is definitely something we need to get fixed ASAP, thank you roman_g for your analysis and summary
14:36:33 <roman_g> I was offline probably. Will look at the logs.
14:36:44 <mattmceuen> Nope, it's coming up in a half hour :)
14:37:01 <roman_g> Ah, today. Good.
14:37:17 <roman_g> Nishant_: thank you!
14:37:37 <mattmceuen> Shall we see how the discussion goes on the OSH side, and if confirmed to be an OSH issue we can help dev/test as possible?  And otherwise circle back in the chat room roman_g?
14:38:04 <roman_g> Yes, sure. I have a testbed running, and can test easily.
14:38:19 <mattmceuen> sweet
14:38:31 <mattmceuen> #topic Deckhand & Shipyard - OpenSUSE builds
14:38:43 <roman_g> Arun has started to work on building OpenSUSE-based images here: https://review.openstack.org/#/q/status:open+branch:master+topic:airship_suse
14:38:50 <roman_g> Does not align good with https://airship-specs.readthedocs.io/en/latest/specs/approved/multi-linux-distros.html we have approved
14:38:59 <roman_g> #action Need to follow-up with/between Arun and James Gu
14:39:01 <mattmceuen> What is the main difference between the approaches?
14:39:47 <roman_g> Incomplete approach.
14:40:03 <mattmceuen> ok
14:40:12 <roman_g> Not distro-independent, no tests, only image build part
14:40:30 <arunkant> mattmceuen: hi, currently we are working on building images. So adding another variable input to ask for distro flavor and that is used in jobs
14:41:26 <roman_g> arunkant: are those 2 changes complete or WIP?
14:41:30 <arunkant> roman_g: its work-in-progress, so trying to add check and gate jobs for building images. Is there any specific tests are needed other than adding jobs ?
14:42:35 <roman_g> arunkant: check the spec via the link above. There are other things which need to be done.
14:43:14 <roman_g> Also please don't override ubuntu-based :latest with opensuse-based :latest image on quay
14:43:16 <arunkant> roman_g: Also need clarification where the docs need to be provided for building opensuse images . Is it updated spec or some other doc section ?
14:43:58 <roman_g> in deckhand and in shipyard docs
14:44:31 <roman_g> spec is used to discuss implementation, collect feedback
14:45:13 <mattmceuen> arunkant, can you please give the spec a read, and see if it covers all the bases or if there are any gaps from your perspective?
14:45:34 <roman_g> Can you reach out to James Gu (SUSE Gmbh.) and may be talk to him on implementation specifics detailed in the spec he wrote?
14:45:47 <arunkant> roman_g: okay. Will look into docs for those components.
14:46:05 <mattmceuen> feel free to shout here in the chat room any time, really awesome to see your work on this, and looking forward to helping.  Thanks for your efforts here
14:46:37 <arunkant> roman_g: Yes, I work with James Gu so will clarify the details with him.
14:47:12 <mattmceuen> Anything else on this topic all?
14:47:29 <roman_g> Actually, my top list of what I would love to see in Airship is to see it working on SLES & RHEL (OpenSUSE & CentOS) in addition to Ubuntu.
14:47:39 <roman_g> So, thank you for your work, arunkant
14:47:40 <arunkant> roman_g: will clarify in chat about the image tag question as oepnsuse images uses different tag (opensuse_15_latest)
14:48:02 <roman_g> arunkant: ahha, good.
14:48:48 <roman_g> mattmceuen
14:49:24 <arunkant> roman_g: yes that's the plan to make airship components to work with opensuse
14:49:44 <mattmceuen> ++
14:50:05 <mattmceuen> #topic Patchset reviews requested
14:50:14 <mattmceuen> https://review.openstack.org/#/c/615892/ - docs: Add use cases for each of the mutation operations
14:50:27 <mattmceuen> Hooray for docs!
14:50:37 <mattmceuen> oh that guy's merged already
14:50:42 <mattmceuen> I should have checked :D
14:51:12 <mattmceuen> Any other patchsets that folks would like to request some solid review?
14:51:42 <mattmceuen> #topic Roundtable
14:51:56 <mattmceuen> Any other topics, all?
14:53:05 <roman_g> Nothing. Thank you, Matt.
14:53:41 <mattmceuen> Alright - thanks everyone for joining us today
14:53:46 <mattmceuen> Have a great week!
14:53:55 <mattmceuen> #endmeeting