| *** aeng has quit IRC | 00:04 | |
| *** esikachev has quit IRC | 00:04 | |
| *** ijw has joined #openstack-infra | 00:04 | |
| *** pahuang has quit IRC | 00:05 | |
| *** ijw has quit IRC | 00:10 | |
| *** ijw has joined #openstack-infra | 00:10 | |
| *** dingyichen has joined #openstack-infra | 00:11 | |
| *** dimtruck is now known as zz_dimtruck | 00:12 | |
| *** sflanigan has joined #openstack-infra | 00:14 | |
| *** ijw has quit IRC | 00:15 | |
| *** aeng has joined #openstack-infra | 00:17 | |
| *** zz_dimtruck is now known as dimtruck | 00:19 | |
| *** pahuang has joined #openstack-infra | 00:21 | |
| openstackgerrit | Jesse Keating proposed openstack-infra/zuul feature/zuulv3: support github pull request labels https://review.openstack.org/444511 | 00:22 |
|---|---|---|
| openstackgerrit | Jesse Keating proposed openstack-infra/zuul feature/zuulv3: Allow github trigger to match on branches/refs https://review.openstack.org/445625 | 00:22 |
| openstackgerrit | Jesse Keating proposed openstack-infra/zuul feature/zuulv3: Set filter according to PR/Change in URL https://review.openstack.org/446782 | 00:22 |
| openstackgerrit | Jesse Keating proposed openstack-infra/zuul feature/zuulv3: Support for dependent pipelines with github https://review.openstack.org/445292 | 00:22 |
| openstackgerrit | Jesse Keating proposed openstack-infra/zuul feature/zuulv3: Add basic Github Zuul Reporter. https://review.openstack.org/443323 | 00:22 |
| openstackgerrit | Jesse Keating proposed openstack-infra/zuul feature/zuulv3: Configurable SSH access to GitHub https://review.openstack.org/444034 | 00:22 |
| openstackgerrit | Jesse Keating proposed openstack-infra/zuul feature/zuulv3: Add 'push' and 'tag' github webhook events. https://review.openstack.org/443947 | 00:22 |
| openstackgerrit | Jesse Keating proposed openstack-infra/zuul feature/zuulv3: Add 'pr-comment' github webhook event https://review.openstack.org/443959 | 00:22 |
| openstackgerrit | Jesse Keating proposed openstack-infra/zuul feature/zuulv3: Support for github commit status https://review.openstack.org/444060 | 00:22 |
| openstackgerrit | Jesse Keating proposed openstack-infra/zuul feature/zuulv3: Better merge message for GitHub pull reqeusts https://review.openstack.org/445644 | 00:22 |
| openstackgerrit | Jesse Keating proposed openstack-infra/zuul feature/zuulv3: Test gerrit and github drivers in same tenant https://review.openstack.org/448257 | 00:22 |
| openstackgerrit | Jesse Keating proposed openstack-infra/zuul feature/zuulv3: Encapsulate determining the event purpose https://review.openstack.org/445242 | 00:22 |
| openstackgerrit | Jesse Keating proposed openstack-infra/zuul feature/zuulv3: Allow using webapp from connections https://review.openstack.org/439831 | 00:22 |
| openstackgerrit | Jesse Keating proposed openstack-infra/zuul feature/zuulv3: Merge pull requests from github reporter https://review.openstack.org/444463 | 00:22 |
| openstackgerrit | Jesse Keating proposed openstack-infra/zuul feature/zuulv3: Support GitHub PR webhooks https://review.openstack.org/439834 | 00:22 |
| openstackgerrit | Jesse Keating proposed openstack-infra/zuul feature/zuulv3: GitHub file matching support https://review.openstack.org/446113 | 00:22 |
| openstackgerrit | Jesse Keating proposed openstack-infra/zuul feature/zuulv3: Log GitHub API rate limit https://review.openstack.org/446150 | 00:22 |
| *** jaypipes has quit IRC | 00:23 | |
| *** gildub has joined #openstack-infra | 00:23 | |
| *** rlandy has quit IRC | 00:33 | |
| *** Sukhdev has quit IRC | 00:33 | |
| *** pvaneck has quit IRC | 00:37 | |
| *** rhallisey has quit IRC | 00:38 | |
| openstackgerrit | Rosario Di Somma proposed openstack-infra/shade master: Use requests-mock for the volume backup tests https://review.openstack.org/459898 | 00:39 |
| *** nicolasbock has joined #openstack-infra | 00:41 | |
| *** cuongnv has joined #openstack-infra | 00:43 | |
| *** arif-ali has quit IRC | 00:45 | |
| *** arif-ali has joined #openstack-infra | 00:46 | |
| *** Apoorva_ has joined #openstack-infra | 00:49 | |
| *** Apoorva_ has quit IRC | 00:49 | |
| *** Apoorva_ has joined #openstack-infra | 00:50 | |
| *** Apoorva_ has quit IRC | 00:52 | |
| dmsimard | If any cores is around, would appreciate a review on https://review.openstack.org/#/c/459462/ | 00:52 |
| *** Apoorva has quit IRC | 00:52 | |
| *** zerick has quit IRC | 00:55 | |
| *** jamesmca_ has joined #openstack-infra | 00:55 | |
| *** zerick has joined #openstack-infra | 00:57 | |
| *** esikachev has joined #openstack-infra | 00:57 | |
| *** deafullt has quit IRC | 00:58 | |
| *** watersoul_ has joined #openstack-infra | 00:59 | |
| *** MasterOfBugs has quit IRC | 00:59 | |
| *** watersoul has quit IRC | 01:00 | |
| *** GregHouse has quit IRC | 01:00 | |
| *** dmellado has quit IRC | 01:00 | |
| *** _ari_ has quit IRC | 01:00 | |
| *** jamesmca_ has quit IRC | 01:00 | |
| *** _ari_ has joined #openstack-infra | 01:00 | |
| *** stevebaker has quit IRC | 01:01 | |
| *** jlvillal has quit IRC | 01:01 | |
| *** GregHouse has joined #openstack-infra | 01:01 | |
| *** esikachev has quit IRC | 01:01 | |
| *** stevebaker has joined #openstack-infra | 01:02 | |
| *** dmellado has joined #openstack-infra | 01:02 | |
| *** Julien-zte has joined #openstack-infra | 01:03 | |
| *** yuanying_ has joined #openstack-infra | 01:04 | |
| *** oomichi has quit IRC | 01:05 | |
| *** yuanying has quit IRC | 01:06 | |
| *** jlvillal has joined #openstack-infra | 01:06 | |
| *** oomichi has joined #openstack-infra | 01:07 | |
| *** xingchao has joined #openstack-infra | 01:11 | |
| *** wznoinsk has quit IRC | 01:14 | |
| *** migi has quit IRC | 01:14 | |
| ianw | clarkb: any memory of what the "reload" bit for tls/devstack was in this -> https://review.openstack.org/459904 | 01:14 |
| *** timothyb89 has quit IRC | 01:14 | |
| *** migi has joined #openstack-infra | 01:14 | |
| *** migi has quit IRC | 01:14 | |
| *** migi has joined #openstack-infra | 01:14 | |
| *** tjones has joined #openstack-infra | 01:14 | |
| *** timothyb89 has joined #openstack-infra | 01:14 | |
| *** wznoinsk has joined #openstack-infra | 01:14 | |
| *** rfolco has quit IRC | 01:17 | |
| dmsimard | ianw: thank you sir | 01:18 |
| *** tjones has quit IRC | 01:19 | |
| *** lindsayk has quit IRC | 01:20 | |
| *** tdasilva has quit IRC | 01:21 | |
| clarkb | ianw: just what the comment says | 01:22 |
| clarkb | restart failed on older apache in trusty and centos. centos has since ficed | 01:23 |
| clarkb | *fixed | 01:23 |
| clarkb | trusty I dont think we care about anymore | 01:23 |
| *** sflanigan has quit IRC | 01:23 | |
| ianw | clarkb: ok ... well yeah, as that change mentions, it seems like mod_wsgi isn't graceful reload safe | 01:23 |
| openstackgerrit | Merged openstack-infra/project-config master: Adjust OSAD integration job in ARA check gate to make it work https://review.openstack.org/459462 | 01:24 |
| ianw | so we just replaced one problem, that requires a reload, with another, that required *not* to reload | 01:24 |
| clarkb | :) it should be safe to restart now | 01:24 |
| ianw | clarkb: if you wouldn't mind dropping a comment with your recollection as above for the other reviewers, that would be great | 01:25 |
| clarkb | since centos package update magically fixed it and we qrent testing on trusty | 01:25 |
| clarkb | kk | 01:25 |
| *** stevebaker has quit IRC | 01:25 | |
| *** tdasilva has joined #openstack-infra | 01:25 | |
| *** stevebaker has joined #openstack-infra | 01:26 | |
| *** deafult has joined #openstack-infra | 01:26 | |
| *** mriedem has quit IRC | 01:28 | |
| *** thorst_afk has joined #openstack-infra | 01:30 | |
| *** tjones has joined #openstack-infra | 01:31 | |
| *** ijw has joined #openstack-infra | 01:32 | |
| *** wolverineav has quit IRC | 01:35 | |
| *** EricGonczer_ has joined #openstack-infra | 01:41 | |
| *** baoli has quit IRC | 01:42 | |
| *** aeng has quit IRC | 01:43 | |
| *** liujiong has joined #openstack-infra | 01:45 | |
| *** tonytan_brb has quit IRC | 01:47 | |
| *** tonytan4ever has joined #openstack-infra | 01:47 | |
| *** yanyanhu has joined #openstack-infra | 01:47 | |
| *** ijw has quit IRC | 01:47 | |
| *** sflanigan has joined #openstack-infra | 01:48 | |
| *** tphummel has quit IRC | 01:52 | |
| *** hongbin has joined #openstack-infra | 01:53 | |
| *** mat128 has joined #openstack-infra | 01:54 | |
| *** mat128 has quit IRC | 01:55 | |
| *** aeng has joined #openstack-infra | 01:57 | |
| *** thorst_afk has quit IRC | 02:05 | |
| *** MasterOfBugs has joined #openstack-infra | 02:05 | |
| *** dhinesh has joined #openstack-infra | 02:06 | |
| *** mat128 has joined #openstack-infra | 02:07 | |
| *** mat128 has quit IRC | 02:10 | |
| *** baoli has joined #openstack-infra | 02:13 | |
| *** esikachev has joined #openstack-infra | 02:14 | |
| *** yamahata_ has quit IRC | 02:15 | |
| dmsimard | ianw: ugh, for https://review.openstack.org/#/c/459462/ it fixed for Ubuntu but CentOS OpenStack Ansible requires EPEL to be enabled because they have something in bindep that requires it :( | 02:16 |
| dmsimard | I wonder if bindep could have logic around epel | 02:16 |
| *** esikachev has quit IRC | 02:19 | |
| *** EricGonczer_ has quit IRC | 02:19 | |
| *** sree has joined #openstack-infra | 02:21 | |
| *** sree has quit IRC | 02:26 | |
| *** tuanluong has joined #openstack-infra | 02:28 | |
| openstackgerrit | David Moreau Simard proposed openstack-infra/project-config master: Enable EPEL for openstack-ansible CentOS bindeps packages https://review.openstack.org/459913 | 02:30 |
| dmsimard | I hate this trial and error around job iteration | 02:30 |
| dmsimard | Can't wait for zuul v3 :) | 02:30 |
| *** ijw_ has joined #openstack-infra | 02:33 | |
| *** zhurong has joined #openstack-infra | 02:33 | |
| *** ijw_ has quit IRC | 02:37 | |
| *** esikachev has joined #openstack-infra | 02:38 | |
| *** nicolasbock has quit IRC | 02:41 | |
| *** esikachev has quit IRC | 02:42 | |
| *** sflanigan has quit IRC | 02:43 | |
| *** adrian_otto has joined #openstack-infra | 02:45 | |
| *** stakeda has joined #openstack-infra | 02:46 | |
| *** baoli has quit IRC | 02:47 | |
| *** baoli has joined #openstack-infra | 02:47 | |
| *** dchen has joined #openstack-infra | 02:47 | |
| ianw | dmsimard: hmmm ... yeah i really do want to avoid depending on it unnecessarily | 02:48 |
| dmsimard | Random thought, [platform:epel] could enable epel, install all the packages for that platform and disable epel ? | 02:48 |
| dmsimard | Not familiar at all with the internals of bindep though | 02:49 |
| ianw | dmsimard: yeah, it's not a bad idea -- it would just need to --enablerepo epel probably | 02:49 |
| *** baoli has quit IRC | 02:49 | |
| dmsimard | A bit like my patch here :p https://review.openstack.org/459913 | 02:49 |
| dmsimard | Oh, with --enablerepo would work too I guess | 02:50 |
| *** tphummel has joined #openstack-infra | 02:51 | |
| ianw | i think i quite like that platform epel idea | 02:52 |
| *** gildub has quit IRC | 02:55 | |
| *** gcb has joined #openstack-infra | 02:55 | |
| *** sflanigan has joined #openstack-infra | 02:55 | |
| *** sflanigan has joined #openstack-infra | 02:55 | |
| *** erikmwilson has joined #openstack-infra | 03:00 | |
| *** yamahata_ has joined #openstack-infra | 03:01 | |
| *** esikachev has joined #openstack-infra | 03:05 | |
| *** thorst_afk has joined #openstack-infra | 03:06 | |
| *** adrian_otto has quit IRC | 03:09 | |
| *** thorst_afk has quit IRC | 03:10 | |
| *** jamesmca_ has joined #openstack-infra | 03:11 | |
| *** Douhet has quit IRC | 03:13 | |
| *** zhurong has quit IRC | 03:15 | |
| *** jamesmca_ has quit IRC | 03:16 | |
| *** esikachev has quit IRC | 03:16 | |
| *** dimtruck is now known as zz_dimtruck | 03:17 | |
| *** dhajare has joined #openstack-infra | 03:18 | |
| *** Douhet has joined #openstack-infra | 03:19 | |
| *** psachin has joined #openstack-infra | 03:20 | |
| *** vhosakot has quit IRC | 03:22 | |
| *** amotoki has joined #openstack-infra | 03:27 | |
| *** tjones has left #openstack-infra | 03:27 | |
| yamamoto | something changed recently in iptables rules on test VMs? | 03:29 |
| yamamoto | it seems empty http://logs.openstack.org/87/199387/70/check/gate-tempest-dsvm-networking-midonet-v2-ubuntu-xenial/abd4e3b/logs/iptables.txt.gz | 03:30 |
| yamamoto | ie. no openstack-INPUT chain etc | 03:30 |
| *** vsaienk0 has quit IRC | 03:33 | |
| clarkb | yamamoto: we changed how the rules are aplied when we build thr images but pabelanger and cmurphy checked them today and thought they were equivalent | 03:33 |
| clarkb | ianw: might be able to double check | 03:33 |
| *** aeng has quit IRC | 03:34 | |
| *** sflanigan has quit IRC | 03:36 | |
| *** thorst_afk has joined #openstack-infra | 03:37 | |
| *** vikrant has joined #openstack-infra | 03:37 | |
| *** salv-orlando has joined #openstack-infra | 03:38 | |
| *** cuongnv has quit IRC | 03:40 | |
| *** cuongnv has joined #openstack-infra | 03:41 | |
| *** salv-orlando has quit IRC | 03:43 | |
| *** Julien-zte has quit IRC | 03:45 | |
| *** ihrachys has quit IRC | 03:46 | |
| *** Julien-zte has joined #openstack-infra | 03:46 | |
| *** vsaienk0 has joined #openstack-infra | 03:49 | |
| *** aeng has joined #openstack-infra | 03:51 | |
| yamamoto | clarkb: our (networking-midonet) jobs are failing because of missing chains | 03:52 |
| *** sflanigan has joined #openstack-infra | 03:52 | |
| yamamoto | https://bugs.launchpad.net/networking-midonet/+bug/1686271 | 03:53 |
| openstack | Launchpad bug 1686271 in networking-midonet "all tempest jobs are failing" [Critical,In progress] - Assigned to YAMAMOTO Takashi (yamamoto) | 03:53 |
| *** amotoki has quit IRC | 03:54 | |
| ianw | yamamoto: reading, hang on ... | 03:55 |
| *** gildub has joined #openstack-infra | 03:55 | |
| yamamoto | our gate hook tweaks rules to allow some node-local traffic which is used by midonet agent. it's assuming the old structure of rules and thus failing. we can tweak the script if desirable. | 03:57 |
| *** ykarel has joined #openstack-infra | 03:59 | |
| *** faizy has joined #openstack-infra | 03:59 | |
| *** thorst_afk has quit IRC | 04:00 | |
| ianw | http://paste.openstack.org/show/607981/ | 04:00 |
| ianw | hmm | 04:00 |
| ianw | urgh, this looks like a loop -> http://paste.openstack.org/show/607982/ | 04:01 |
| *** faizy_ has quit IRC | 04:02 | |
| *** gouthamr has quit IRC | 04:02 | |
| yamamoto | heh | 04:02 |
| ianw | yamamoto: this would be a bug in the images | 04:03 |
| *** tinwood has quit IRC | 04:05 | |
| *** david-lyle has quit IRC | 04:05 | |
| *** david-lyle_ has joined #openstack-infra | 04:05 | |
| *** tinwood has joined #openstack-infra | 04:06 | |
| yamamoto | thank you. will it likely be fixed soon, or should we prepare some short-term workaround? | 04:06 |
| yamamoto | we==affected projects like networking-midonet | 04:08 |
| *** eeiden has quit IRC | 04:09 | |
| *** amotoki has joined #openstack-infra | 04:10 | |
| *** david-lyle_ is now known as david-lyle | 04:12 | |
| *** hughhalf has quit IRC | 04:13 | |
| *** hughhalf has joined #openstack-infra | 04:13 | |
| ianw | yamamoto: ahh, how soon is "soon" to you :) | 04:14 |
| yamamoto | a few days? | 04:14 |
| pabelanger | ianw: I think I know the issue | 04:15 |
| ianw | pabelanger: yeah, change coming ... that link isn't required | 04:15 |
| pabelanger | I think we setup out symlinkx backwards in project-config | 04:15 |
| *** amotoki has quit IRC | 04:15 | |
| *** erikmwilson has quit IRC | 04:16 | |
| pabelanger | ianw: ya, we set it up one way in DIB, then puppet turns it around the other | 04:16 |
| *** erikmwilson has joined #openstack-infra | 04:16 | |
| pabelanger | so we symlink to each | 04:16 |
| ianw | pabelanger: well, there seem to be a number of issues ... | 04:16 |
| ianw | i don't think iptables-persistent is a service | 04:17 |
| ianw | it's netfilter-persistent now | 04:17 |
| ianw | oh, hang on, i see that's a branch, that's all good | 04:18 |
| *** Julien-zte has quit IRC | 04:19 | |
| ianw | yamamoto: we're talking image build time and upload, so more like hours, but certainly not days | 04:19 |
| yamamoto | ok, thank you | 04:19 |
| *** amotoki has joined #openstack-infra | 04:20 | |
| *** Julien-zte has joined #openstack-infra | 04:20 | |
| *** psachin has quit IRC | 04:21 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/project-config master: Fix symlink for iptables rules https://review.openstack.org/459939 | 04:21 |
| pabelanger | ianw: ^should be the fix | 04:21 |
| pabelanger | also, we can pause image builds if needed, then delete the curent xenial image | 04:21 |
| pabelanger | that should unblock the gate | 04:21 |
| ianw | pabelanger: why do we want the symlink at all? | 04:22 |
| ianw | the netfilter-persistent package does not seem to want /etc/iptables/rules | 04:22 |
| pabelanger | until we drop puppet-iptables, we'll need to keep it | 04:22 |
| ianw | it by default just drops the .v4 and .v6 files | 04:22 |
| pabelanger | but agree, we can likely remove it after we unpuppet | 04:22 |
| ianw | oohhh, so this hasn't actually been removed from puppet yet? | 04:23 |
| *** psachin has joined #openstack-infra | 04:23 | |
| pabelanger | right | 04:23 |
| pabelanger | we need to land https://review.openstack.org/#/c/458986/ | 04:23 |
| pabelanger | and parent change | 04:23 |
| pabelanger | which, we can if you want | 04:23 |
| pabelanger | I tested ubuntu-xenial today, and iptables was correct | 04:24 |
| ianw | ahh, well why don't we land that, and just create the two files | 04:24 |
| pabelanger | you call :) | 04:25 |
| pabelanger | I won't be able to support for the next 8-9 hours, but in my morning I can | 04:25 |
| pabelanger | ianw: we'll need https://review.openstack.org/#/c/458944/ too | 04:27 |
| *** tuanluong_ has joined #openstack-infra | 04:28 | |
| pabelanger | ianw: once they are merged into system-config, you'd be able to delete current ubuntu-xenial dibs, which reverts to yesterday images, and start new builds will puppet-iptables removed | 04:29 |
| openstackgerrit | Ian Wienand proposed openstack-infra/project-config master: Don't create /etc/iptables/rules https://review.openstack.org/459939 | 04:31 |
| ianw | pabelanger: ^ ok, so you'd agree with that revision? | 04:32 |
| *** tuanluong has quit IRC | 04:32 | |
| pabelanger | ianw: why did we symlink to begin with? | 04:34 |
| ianw | ? | 04:34 |
| *** tuanluong__ has joined #openstack-infra | 04:34 | |
| pabelanger | for puppet-iptables, why the symlink at all? | 04:34 |
| pabelanger | I never understood it | 04:34 |
| ianw | oh yeah, i mean i don't know | 04:34 |
| ianw | maybe before iptables-persistent understood ipv6? | 04:35 |
| pabelanger | possible | 04:37 |
| ianw | i just checked even the iptables-persistent source from trusty, and that's looking for .v4 file | 04:37 |
| ianw | http://mirrors.kernel.org/ubuntu/pool/universe/i/iptables-persistent/iptables-persistent_0.5.7_all.deb | 04:37 |
| pabelanger | k, +2 | 04:38 |
| pabelanger | but we'll need to +3 458944 first | 04:38 |
| *** tuanluong_ has quit IRC | 04:38 | |
| pabelanger | to ensure puppet-iptables is removed | 04:38 |
| pabelanger | then delete and build can happen | 04:38 |
| pabelanger | we are protected on multi nodes jobs for now, since we setup iptables rules | 04:39 |
| pabelanger | and nodepool fails to bring the node online | 04:39 |
| *** salv-orlando has joined #openstack-infra | 04:39 | |
| pabelanger | we likely need also delete trusty too | 04:40 |
| pabelanger | ya, just confirmed it also has bad rules | 04:41 |
| ianw | ok, let's get these changes in | 04:43 |
| *** salv-orlando has quit IRC | 04:44 | |
| *** hongbin has quit IRC | 04:44 | |
| openstackgerrit | Merged openstack-infra/system-config master: Move afs client to openstack_project::server https://review.openstack.org/458944 | 04:45 |
| *** gongysh has joined #openstack-infra | 04:46 | |
| openstackgerrit | Merged openstack-infra/system-config master: Move iptables management to o_p::server https://review.openstack.org/458986 | 04:47 |
| openstackgerrit | Merged openstack-infra/project-config master: Don't create /etc/iptables/rules https://review.openstack.org/459939 | 04:48 |
| *** gongysh has quit IRC | 04:49 | |
| *** jascott1 has quit IRC | 04:49 | |
| *** amotoki has quit IRC | 04:49 | |
| pabelanger | ianw: okay, I'm going to grab some shut-eye. Suspect I'll be up early to see how you make out with rebuilds | 04:50 |
| ianw | pabelanger: ok, i'll delete and restart builds now | 04:50 |
| pabelanger | kk | 04:51 |
| *** eeiden has joined #openstack-infra | 04:51 | |
| *** tnovacik has joined #openstack-infra | 04:52 | |
| *** faizy has quit IRC | 04:54 | |
| *** tuanluong has joined #openstack-infra | 04:55 | |
| *** thorst_afk has joined #openstack-infra | 04:57 | |
| *** sree has joined #openstack-infra | 04:57 | |
| *** tuanluong_ has joined #openstack-infra | 04:57 | |
| *** amotoki has joined #openstack-infra | 04:58 | |
| *** tuanluong__ has quit IRC | 04:58 | |
| *** tuanluong has quit IRC | 05:00 | |
| *** gongysh has joined #openstack-infra | 05:01 | |
| *** thorst_afk has quit IRC | 05:02 | |
| *** claudiub has joined #openstack-infra | 05:03 | |
| *** cshastri has joined #openstack-infra | 05:07 | |
| *** zhurong has joined #openstack-infra | 05:08 | |
| *** jamesmca_ has joined #openstack-infra | 05:11 | |
| *** nadya has joined #openstack-infra | 05:15 | |
| *** lamt has quit IRC | 05:15 | |
| *** jamesmca_ has quit IRC | 05:16 | |
| *** salv-orlando has joined #openstack-infra | 05:16 | |
| *** jascott1 has joined #openstack-infra | 05:17 | |
| *** nadya has quit IRC | 05:21 | |
| *** dhinesh has quit IRC | 05:22 | |
| *** tnovacik has quit IRC | 05:23 | |
| *** yolanda has quit IRC | 05:28 | |
| *** pgadiya has joined #openstack-infra | 05:28 | |
| *** darrenc has quit IRC | 05:28 | |
| *** darrenc has joined #openstack-infra | 05:28 | |
| *** r1chardj0n3s has joined #openstack-infra | 05:29 | |
| *** sree has quit IRC | 05:31 | |
| *** e0ne has joined #openstack-infra | 05:32 | |
| *** e0ne has quit IRC | 05:33 | |
| *** florianf has joined #openstack-infra | 05:35 | |
| *** sflanigan has quit IRC | 05:35 | |
| *** aeng has quit IRC | 05:35 | |
| *** cshastri has quit IRC | 05:36 | |
| *** dingyichen has quit IRC | 05:36 | |
| *** gouthamr has joined #openstack-infra | 05:36 | |
| *** alexchadin has joined #openstack-infra | 05:40 | |
| *** sree has joined #openstack-infra | 05:40 | |
| *** cshastri has joined #openstack-infra | 05:41 | |
| *** pahuang has quit IRC | 05:42 | |
| *** Hal has quit IRC | 05:44 | |
| *** prateek has joined #openstack-infra | 05:46 | |
| *** aeng has joined #openstack-infra | 05:48 | |
| *** adrian_otto has joined #openstack-infra | 05:48 | |
| *** cody-somerville has quit IRC | 05:49 | |
| *** cody-somerville has joined #openstack-infra | 05:49 | |
| *** dingyichen has joined #openstack-infra | 05:49 | |
| *** csomerville has joined #openstack-infra | 05:50 | |
| *** zhurong has quit IRC | 05:52 | |
| *** cody-somerville has quit IRC | 05:53 | |
| *** pahuang has joined #openstack-infra | 05:55 | |
| *** thorst_afk has joined #openstack-infra | 05:58 | |
| *** adriant has quit IRC | 06:00 | |
| *** thorst_afk has quit IRC | 06:02 | |
| *** alexchadin has quit IRC | 06:06 | |
| *** tphummel has quit IRC | 06:08 | |
| *** esikachev has joined #openstack-infra | 06:13 | |
| ianw | ok, we got a few new xenial images uploaded | 06:14 |
| *** nadya has joined #openstack-infra | 06:14 | |
| *** xingchao has quit IRC | 06:15 | |
| *** xingchao has joined #openstack-infra | 06:15 | |
| *** chenying_ has quit IRC | 06:16 | |
| *** chenying_ has joined #openstack-infra | 06:16 | |
| *** sshnaidm has quit IRC | 06:16 | |
| ianw | ok, new node with what looks like right rules to me -> http://paste.openstack.org/show/607992/ | 06:22 |
| ianw | multinode jobs seem to be processing again | 06:23 |
| *** sshnaidm has joined #openstack-infra | 06:24 | |
| *** andreas_s has joined #openstack-infra | 06:24 | |
| *** zhurong has joined #openstack-infra | 06:25 | |
| *** amotoki has quit IRC | 06:30 | |
| *** amotoki has joined #openstack-infra | 06:31 | |
| *** kzaitsev_ws has joined #openstack-infra | 06:32 | |
| *** armax has quit IRC | 06:33 | |
| *** zhurong has quit IRC | 06:33 | |
| *** ijw has joined #openstack-infra | 06:36 | |
| cmurphy | thanks ianw | 06:39 |
| *** d0ugal has quit IRC | 06:39 | |
| *** ijw has quit IRC | 06:40 | |
| *** d0ugal has joined #openstack-infra | 06:41 | |
| *** makowals has quit IRC | 06:44 | |
| *** tie has joined #openstack-infra | 06:44 | |
| *** makowals has joined #openstack-infra | 06:44 | |
| *** nadya has quit IRC | 06:45 | |
| *** amotoki has quit IRC | 06:47 | |
| *** yolanda has joined #openstack-infra | 06:49 | |
| *** iMe has joined #openstack-infra | 06:51 | |
| *** matrohon has joined #openstack-infra | 06:52 | |
| *** rossella_s has joined #openstack-infra | 06:55 | |
| *** iMe has quit IRC | 06:58 | |
| *** thorst_afk has joined #openstack-infra | 06:59 | |
| *** MasterOfBugs has quit IRC | 07:00 | |
| *** namnh has joined #openstack-infra | 07:03 | |
| *** thorst_afk has quit IRC | 07:03 | |
| *** tesseract has joined #openstack-infra | 07:04 | |
| *** pgadiya has quit IRC | 07:04 | |
| *** gvrangan_ has joined #openstack-infra | 07:05 | |
| gvrangan_ | hi test folks | 07:05 |
| gvrangan_ | we are having problems with running packstack Ocata | 07:05 |
| *** shardy has joined #openstack-infra | 07:05 | |
| gvrangan_ | repeated db issues keep popping up | 07:05 |
| gvrangan_ | any suggestions? | 07:05 |
| *** isaacb has joined #openstack-infra | 07:06 | |
| ianw | gvrangan_: probably going to need more details. what do you mean db issues? | 07:06 |
| *** e0ne has joined #openstack-infra | 07:07 | |
| ianw | cmurphy: np ... keeping an eye but things seem to be moving | 07:08 |
| ianw | 509 trusty mostly uploaded too | 07:09 |
| gvrangan_ | ERROR oslo_db.sqlalchemy.exc_filters [req-40ba0807-1b58-49a2-a0b5-e565138a095d - - - - -] DBAPIError exception wrapped from (pymysql.err.InternalError) (1054, u"Unknown column 'aggregates.uuid' in 'field list'") [SQL: u'SELECT aggregates.created_at AS aggregates_created_at, aggregates.updated_at AS aggregates_updated_at, aggregates.deleted_at AS aggregates_deleted_at, aggregates.deleted AS aggregates_deleted, aggregates.id AS aggregates_id, | 07:10 |
| gvrangan_ | aggregates.uuid AS aggregates_uuid, aggregates.name AS aggregates_name, aggregate_hosts_1.created_at AS aggregate_hosts_1_created_at, aggregate_hosts_1.updated_at AS aggregate_hosts_1_updated_at, aggregate_hosts_1.deleted_at AS aggregate_hosts_1_deleted_at, aggregate_hosts_1.deleted AS aggregate_hosts_1_deleted, aggregate_hosts_1.id AS aggregate_hosts_1_id, aggregate_hosts_1.host AS aggregate_hosts_1_host, aggregate_hosts_1.aggregate_id AS | 07:10 |
| gvrangan_ | aggregate_hosts_1_aggregate_id, aggregate_metadata_1.created_at AS aggregate_metadata_1_created_at, aggregate_metadata_1.updated_at AS aggregate_metadata_1_updated_at, aggregate_metadata_1.deleted_at AS aggregate_metadata_1_deleted_at, aggregate_metadata_1.deleted AS aggregate_metadata_1_deleted, aggregate_metadata_1.id AS aggregate_metadata_1_id, aggregate_metadata_1.`key` AS aggregate_metadata_1_key, aggregate_metadata_1.value AS | 07:10 |
| gvrangan_ | aggregate_metadata_1_value, aggregate_metadata_1.aggregate_id AS aggregate_metadata_1_aggregate_id \nFROM aggregates LEFT OUTER JOIN aggregate_hosts AS aggregate_hosts_1 ON aggregates.id = aggregate_hosts_1.aggregate_id AND aggregate_hosts_1.deleted = %(deleted_1)s AND aggregates.deleted = %(deleted_2)s LEFT OUTER JOIN aggregate_metadata AS aggregate_metadata_1 ON aggregates.id = aggregate_metadata_1.aggregate_id AND aggregate_metadata_1.dele | 07:10 |
| gvrangan_ | ted = %(deleted_3)s AND aggregates.deleted = %(deleted_4)s \nWHERE aggregates.deleted = %(deleted_5)s'] [parameters: {u'deleted_5': 0, u'deleted_4': 0, u'deleted_3': 0, u'deleted_2': 0, u'deleted_1': 0}] | 07:10 |
| gvrangan_ | WE had to make number of workers to "1" then faced this issue | 07:11 |
| gvrangan_ | not sure if they are releated | 07:11 |
| gvrangan_ | ianw: thanks for responding | 07:12 |
| *** jamesmca_ has joined #openstack-infra | 07:12 | |
| gvrangan_ | ianw: ^^^ the above are the errors | 07:12 |
| *** lewo has joined #openstack-infra | 07:16 | |
| *** jamesmca_ has quit IRC | 07:16 | |
| *** nadya has joined #openstack-infra | 07:18 | |
| *** pcaruana has joined #openstack-infra | 07:19 | |
| *** Kevin_Zheng has joined #openstack-infra | 07:21 | |
| *** nadya has quit IRC | 07:24 | |
| *** yolanda has quit IRC | 07:25 | |
| *** zhurong has joined #openstack-infra | 07:28 | |
| *** cshastri has quit IRC | 07:29 | |
| *** cshastri has joined #openstack-infra | 07:29 | |
| *** arxcruz has quit IRC | 07:29 | |
| *** arxcruz has joined #openstack-infra | 07:31 | |
| *** adrian_otto has quit IRC | 07:31 | |
| openstackgerrit | Colleen Murphy proposed openstack-infra/system-config master: Clean up puppetmaster cruft https://review.openstack.org/452558 | 07:32 |
| *** arxcruz has quit IRC | 07:32 | |
| *** arxcruz has joined #openstack-infra | 07:32 | |
| *** electrofelix has joined #openstack-infra | 07:34 | |
| *** ralonsoh has joined #openstack-infra | 07:36 | |
| *** vsaienk0 has quit IRC | 07:38 | |
| AJaeger | gvrangan_: please use paste.openstack.org | 07:38 |
| *** jpena|off is now known as jpena | 07:38 | |
| *** efoley has joined #openstack-infra | 07:39 | |
| *** yolanda has joined #openstack-infra | 07:41 | |
| *** jpich has joined #openstack-infra | 07:42 | |
| gvrangan_ | AJaeger, http://paste.openstack.org/show/608000/ | 07:44 |
| gvrangan_ | The nova-scheduler seems to fail in db synbc | 07:44 |
| gvrangan_ | and the packstack halts | 07:44 |
| AJaeger | gvrangan_: #openstack-infra is the wrong channel for this, better ask on #openstack | 07:45 |
| AJaeger | gvrangan_: this channel is about the infrastructure used to build OpenStack - like OpenStack CI | 07:45 |
| openstackgerrit | Arx Cruz proposed openstack-infra/tripleo-ci master: Skipping build image for periodic tempest jobs https://review.openstack.org/459981 | 07:45 |
| gvrangan_ | AJaeger, I used packstack | 07:46 |
| gvrangan_ | hence asked the query here | 07:46 |
| AJaeger | gvrangan_: where does packstack point to openstack-infra? | 07:47 |
| *** gouthamr has quit IRC | 07:47 | |
| gvrangan_ | AJaeger, Sorry will raise in puppet-openstack and openstack channels | 07:47 |
| *** chem has quit IRC | 07:48 | |
| AJaeger | gvrangan_: https://wiki.openstack.org/wiki/Packstack contains a list of channels to ask - #openstack and #rdo | 07:48 |
| gvrangan_ | AJaeger, Thanks for pointing out the wiki, Will try those channels right away | 07:49 |
| *** dizquierdo has joined #openstack-infra | 07:50 | |
| *** e0ne has quit IRC | 07:53 | |
| *** vsaienk0 has joined #openstack-infra | 07:55 | |
| *** tie has quit IRC | 07:57 | |
| *** tie has joined #openstack-infra | 07:58 | |
| *** amotoki has joined #openstack-infra | 07:58 | |
| *** zzzeek has quit IRC | 08:00 | |
| *** martinkopec has joined #openstack-infra | 08:00 | |
| *** chem has joined #openstack-infra | 08:00 | |
| *** zzzeek has joined #openstack-infra | 08:00 | |
| *** thorst_afk has joined #openstack-infra | 08:00 | |
| *** sree has quit IRC | 08:03 | |
| *** gildub has quit IRC | 08:04 | |
| *** Hal has joined #openstack-infra | 08:08 | |
| *** zhurong has quit IRC | 08:08 | |
| *** lucas-afk is now known as lucasagomes | 08:10 | |
| openstackgerrit | Michal Dulko proposed openstack-infra/irc-meetings master: Add OpenStack-Helm community meeting https://review.openstack.org/459988 | 08:12 |
| *** john-davidge has joined #openstack-infra | 08:13 | |
| *** dtantsur|afk is now known as dtantsur | 08:14 | |
| *** xingchao has quit IRC | 08:14 | |
| *** gongysh has quit IRC | 08:16 | |
| *** e0ne has joined #openstack-infra | 08:18 | |
| *** thorst_afk has quit IRC | 08:19 | |
| *** xingchao_ has joined #openstack-infra | 08:19 | |
| *** pbourke has joined #openstack-infra | 08:23 | |
| *** yolanda has quit IRC | 08:24 | |
| *** dizquierdo has quit IRC | 08:24 | |
| openstackgerrit | mathieu bultel proposed openstack-infra/project-config master: Create tripleo-upgrade project https://review.openstack.org/459762 | 08:26 |
| *** dchen has quit IRC | 08:26 | |
| *** r1chardj0n3s has left #openstack-infra | 08:28 | |
| *** dingyichen has quit IRC | 08:28 | |
| *** florianf has quit IRC | 08:28 | |
| *** jaosorior_away is now known as jaosorior | 08:28 | |
| *** priteau has joined #openstack-infra | 08:29 | |
| *** john-davidge has quit IRC | 08:32 | |
| *** mhickey has joined #openstack-infra | 08:32 | |
| *** sree has joined #openstack-infra | 08:33 | |
| *** jascott1 has quit IRC | 08:34 | |
| *** Julien-zte has quit IRC | 08:35 | |
| *** Julien-zte has joined #openstack-infra | 08:36 | |
| *** derekh has joined #openstack-infra | 08:36 | |
| *** esikachev has quit IRC | 08:38 | |
| *** john-davidge has joined #openstack-infra | 08:39 | |
| *** yolanda has joined #openstack-infra | 08:41 | |
| *** florianf has joined #openstack-infra | 08:46 | |
| *** efoley has quit IRC | 08:46 | |
| *** esikachev has joined #openstack-infra | 08:48 | |
| *** AmitKuhikar has joined #openstack-infra | 08:49 | |
| AmitKuhikar | Hi I running the CICD and few TCs in tempest got failed because of below error | 08:50 |
| AmitKuhikar | No valid backend was found. No weighed backends available | 08:50 |
| AmitKuhikar | Can someone encounter the same issue | 08:50 |
| AmitKuhikar | need help to resolve the same | 08:50 |
| *** namnh has quit IRC | 08:53 | |
| *** AJaeger has quit IRC | 08:57 | |
| *** john-davidge has quit IRC | 08:57 | |
| *** sambetts|afk is now known as sambetts | 09:00 | |
| *** namnh has joined #openstack-infra | 09:01 | |
| *** namnh has quit IRC | 09:03 | |
| *** sheeprine has quit IRC | 09:03 | |
| *** sheeprine has joined #openstack-infra | 09:08 | |
| odyssey4me | project-config cores, I'd appreciate a second review for https://review.openstack.org/458890 | 09:10 |
| *** AJaeger has joined #openstack-infra | 09:10 | |
| *** AJaeger has quit IRC | 09:11 | |
| *** stakeda has quit IRC | 09:12 | |
| *** nadya has joined #openstack-infra | 09:12 | |
| *** jamesmca_ has joined #openstack-infra | 09:13 | |
| openstackgerrit | Merged openstack-infra/groups master: Upgrade Media module to 2.0 https://review.openstack.org/459850 | 09:13 |
| *** AJaeger has joined #openstack-infra | 09:14 | |
| *** jamesmca_ has quit IRC | 09:17 | |
| *** tie has quit IRC | 09:21 | |
| *** yamahata_ has quit IRC | 09:21 | |
| *** zhurong has joined #openstack-infra | 09:22 | |
| *** sree has quit IRC | 09:25 | |
| openstackgerrit | Colleen Murphy proposed openstack-infra/system-config master: Remove install_resolv_conf param from s_u_s https://review.openstack.org/458946 | 09:25 |
| *** dkehn has quit IRC | 09:25 | |
| *** sree has joined #openstack-infra | 09:25 | |
| *** sree has quit IRC | 09:26 | |
| *** yolanda has quit IRC | 09:26 | |
| *** jamesmca_ has joined #openstack-infra | 09:26 | |
| *** john-davidge has joined #openstack-infra | 09:26 | |
| *** AmitKuhikar has quit IRC | 09:30 | |
| *** jamesmca_ has quit IRC | 09:30 | |
| *** kjackal_ has joined #openstack-infra | 09:33 | |
| *** xingchao_ has quit IRC | 09:33 | |
| *** tie has joined #openstack-infra | 09:33 | |
| *** zhurong has quit IRC | 09:37 | |
| openstackgerrit | Sagi Shnaidman proposed openstack-infra/tripleo-ci master: Source ci mirrors file if it exists only https://review.openstack.org/460022 | 09:38 |
| *** xingchao has joined #openstack-infra | 09:40 | |
| *** yolanda has joined #openstack-infra | 09:43 | |
| *** sslypushenko_ has quit IRC | 09:45 | |
| *** john-davidge has quit IRC | 09:48 | |
| *** john-davidge has joined #openstack-infra | 09:48 | |
| *** salv-orl_ has joined #openstack-infra | 09:49 | |
| *** salv-orlando has quit IRC | 09:52 | |
| *** tosky has joined #openstack-infra | 09:52 | |
| *** salv-orl_ has quit IRC | 09:55 | |
| *** salv-orlando has joined #openstack-infra | 09:55 | |
| *** sdague has joined #openstack-infra | 09:55 | |
| *** dkehn has joined #openstack-infra | 09:59 | |
| *** Julien-zte has quit IRC | 10:01 | |
| *** nadya has quit IRC | 10:01 | |
| *** pbourke has quit IRC | 10:05 | |
| *** cuongnv has quit IRC | 10:08 | |
| *** efoley has joined #openstack-infra | 10:12 | |
| *** tuanluong_ has quit IRC | 10:15 | |
| *** thorst_afk has joined #openstack-infra | 10:16 | |
| *** sree has joined #openstack-infra | 10:17 | |
| *** thorst_afk has quit IRC | 10:21 | |
| *** alexchadin has joined #openstack-infra | 10:21 | |
| *** sree has quit IRC | 10:21 | |
| *** dizquierdo has joined #openstack-infra | 10:24 | |
| *** liujiong has quit IRC | 10:26 | |
| *** nicolasbock has joined #openstack-infra | 10:32 | |
| *** nadya has joined #openstack-infra | 10:32 | |
| *** nadya has quit IRC | 10:37 | |
| *** eeiden has quit IRC | 10:37 | |
| *** jascott1 has joined #openstack-infra | 10:39 | |
| *** jascott1 has quit IRC | 10:44 | |
| *** jkilpatr has quit IRC | 10:45 | |
| *** yanyanhu has quit IRC | 10:45 | |
| *** salv-orl_ has joined #openstack-infra | 10:48 | |
| openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci master: TEST: DONT RECHECK or REVIEW: periodic jobs https://review.openstack.org/359215 | 10:51 |
| *** salv-orlando has quit IRC | 10:51 | |
| *** pbourke has joined #openstack-infra | 10:52 | |
| *** eeiden has joined #openstack-infra | 10:52 | |
| *** salv-orl_ has quit IRC | 10:52 | |
| *** xingchao has quit IRC | 10:53 | |
| *** nadya has joined #openstack-infra | 10:54 | |
| *** xingchao has joined #openstack-infra | 10:54 | |
| *** kjackal_ has quit IRC | 10:56 | |
| *** lucasagomes is now known as lucas-hungry | 10:58 | |
| *** xingchao has quit IRC | 10:58 | |
| *** lamt_ has joined #openstack-infra | 10:58 | |
| openstackgerrit | Jiri Stransky proposed openstack-infra/tripleo-ci master: Use mixed release for container upgrade https://review.openstack.org/460061 | 10:58 |
| openstackgerrit | Jiri Stransky proposed openstack-infra/tripleo-ci master: Use mixed release for container upgrade https://review.openstack.org/460061 | 10:59 |
| *** lamt_ has quit IRC | 10:59 | |
| *** rossella_s has quit IRC | 11:04 | |
| *** kjackal_ has joined #openstack-infra | 11:04 | |
| *** rossella_s has joined #openstack-infra | 11:05 | |
| *** jkilpatr has joined #openstack-infra | 11:05 | |
| *** shardy is now known as shardy_lunch | 11:09 | |
| *** martinkopec has quit IRC | 11:10 | |
| *** kjackal_ has quit IRC | 11:11 | |
| *** martinkopec has joined #openstack-infra | 11:12 | |
| *** alexchadin has quit IRC | 11:14 | |
| *** esikachev has quit IRC | 11:16 | |
| *** thorst_afk has joined #openstack-infra | 11:17 | |
| *** yolanda has quit IRC | 11:19 | |
| *** martinkopec has quit IRC | 11:19 | |
| *** jaypipes has joined #openstack-infra | 11:22 | |
| *** martinkopec has joined #openstack-infra | 11:23 | |
| *** esikachev has joined #openstack-infra | 11:23 | |
| *** Hal has quit IRC | 11:24 | |
| *** thorst_afk has quit IRC | 11:25 | |
| *** rossella_s has quit IRC | 11:30 | |
| *** alexchadin has joined #openstack-infra | 11:32 | |
| *** ldnunes has joined #openstack-infra | 11:33 | |
| *** yolanda has joined #openstack-infra | 11:34 | |
| *** thorst_afk has joined #openstack-infra | 11:39 | |
| *** mat128 has joined #openstack-infra | 11:41 | |
| *** rfolco has joined #openstack-infra | 11:42 | |
| *** vsaienk0 has quit IRC | 11:47 | |
| *** gildub has joined #openstack-infra | 11:47 | |
| *** efoley has quit IRC | 11:48 | |
| *** john-davidge has quit IRC | 11:48 | |
| *** john-davidge has joined #openstack-infra | 11:50 | |
| *** rossella_s has joined #openstack-infra | 11:55 | |
| *** john-davidge has quit IRC | 11:55 | |
| *** mat128 has quit IRC | 12:00 | |
| *** alexchadin has quit IRC | 12:01 | |
| *** alexchadin has joined #openstack-infra | 12:01 | |
| sshnaidm | pabelanger, ping | 12:03 |
| *** vsaienk0 has joined #openstack-infra | 12:04 | |
| *** sree has joined #openstack-infra | 12:05 | |
| sshnaidm | pabelanger, wrt mirrors for tripleo-ci, I see repo files on nodepool image CentOS-QEMU-EV.repo, CentOS-Ceph-Jewel.repo - which point to mirror.centos.org instead of rh1 mirror. It breaks all our effort to use AFS mirrors. Do you know why are they there? | 12:05 |
| *** sree has quit IRC | 12:10 | |
| *** lucas-hungry is now known as lucasagomes | 12:11 | |
| *** mat128 has joined #openstack-infra | 12:12 | |
| *** ccamacho is now known as ccamacho|lunch | 12:13 | |
| *** AJaeger has quit IRC | 12:13 | |
| *** dtardivel has joined #openstack-infra | 12:16 | |
| *** alexchadin has quit IRC | 12:16 | |
| *** efoley has joined #openstack-infra | 12:19 | |
| *** efoley_ has joined #openstack-infra | 12:21 | |
| *** sree has joined #openstack-infra | 12:21 | |
| *** ramishra has quit IRC | 12:23 | |
| *** efoley has quit IRC | 12:24 | |
| *** sree has quit IRC | 12:26 | |
| *** trown|outtypewww is now known as trown | 12:26 | |
| *** ramishra has joined #openstack-infra | 12:29 | |
| *** rlandy has joined #openstack-infra | 12:37 | |
| *** jpich has quit IRC | 12:39 | |
| *** shardy_lunch is now known as shardy | 12:39 | |
| *** jpich has joined #openstack-infra | 12:40 | |
| *** AJaeger has joined #openstack-infra | 12:43 | |
| *** dtantsur is now known as dtantsur|brb | 12:44 | |
| *** rhallisey has joined #openstack-infra | 12:44 | |
| *** salv-orl_ has joined #openstack-infra | 12:45 | |
| *** mhickey has quit IRC | 12:45 | |
| *** Hal has joined #openstack-infra | 12:48 | |
| *** srwilkers has joined #openstack-infra | 12:50 | |
| *** gildub has quit IRC | 12:53 | |
| *** AJaeger has quit IRC | 12:55 | |
| *** AJaeger has joined #openstack-infra | 12:56 | |
| *** Julien-zte has joined #openstack-infra | 12:56 | |
| *** jordanP has quit IRC | 12:56 | |
| *** jordanP has joined #openstack-infra | 12:56 | |
| *** ccamacho|lunch has quit IRC | 12:57 | |
| *** gcb has quit IRC | 12:58 | |
| tie | ok, on JJB - how does the following templating parsing look to you: https://pastebin.com/K40FXzkj ? | 12:59 |
| *** salv-orl_ has quit IRC | 12:59 | |
| tie | (basically accessing nested, variable or static dictionary keys | 12:59 |
| *** ihrachys has joined #openstack-infra | 13:00 | |
| *** baoli has joined #openstack-infra | 13:00 | |
| *** jascott1 has joined #openstack-infra | 13:02 | |
| openstackgerrit | Doug Hellmann proposed openstack-infra/project-config master: import downpour https://review.openstack.org/460098 | 13:02 |
| *** mriedem has joined #openstack-infra | 13:03 | |
| *** askb has quit IRC | 13:04 | |
| *** yolanda has quit IRC | 13:06 | |
| *** AJaeger has quit IRC | 13:07 | |
| *** mdrabe has joined #openstack-infra | 13:07 | |
| *** srobert has joined #openstack-infra | 13:07 | |
| *** martinkopec has quit IRC | 13:08 | |
| *** jascott1 has quit IRC | 13:09 | |
| *** martinkopec has joined #openstack-infra | 13:10 | |
| *** esikachev has quit IRC | 13:10 | |
| tie | electrofelix, zaro, waynr: ^^ | 13:12 |
| *** ccamacho has joined #openstack-infra | 13:13 | |
| *** mat128 has quit IRC | 13:14 | |
| *** ZZelle has quit IRC | 13:15 | |
| *** kgiusti has joined #openstack-infra | 13:15 | |
| *** Julien-zte has quit IRC | 13:16 | |
| *** jamesdenton has joined #openstack-infra | 13:16 | |
| *** Goneri has joined #openstack-infra | 13:16 | |
| *** Julien-zte has joined #openstack-infra | 13:16 | |
| *** lamt has joined #openstack-infra | 13:18 | |
| *** cshastri has quit IRC | 13:19 | |
| *** tongli has joined #openstack-infra | 13:20 | |
| *** dprince has joined #openstack-infra | 13:20 | |
| *** gouthamr has joined #openstack-infra | 13:21 | |
| *** pradk has quit IRC | 13:22 | |
| *** cshastri has joined #openstack-infra | 13:22 | |
| *** yolanda has joined #openstack-infra | 13:23 | |
| *** ccamacho has quit IRC | 13:25 | |
| *** jcoufal has joined #openstack-infra | 13:25 | |
| *** ccamacho has joined #openstack-infra | 13:26 | |
| *** john-davidge has joined #openstack-infra | 13:28 | |
| *** zz_dimtruck is now known as dimtruck | 13:28 | |
| *** baoli has quit IRC | 13:29 | |
| *** baoli has joined #openstack-infra | 13:30 | |
| *** xyang1 has joined #openstack-infra | 13:30 | |
| *** gordc has joined #openstack-infra | 13:31 | |
| *** AJaeger has joined #openstack-infra | 13:33 | |
| *** thomasem has left #openstack-infra | 13:35 | |
| *** jaosorior is now known as jaosorior_away | 13:36 | |
| *** martinkopec has quit IRC | 13:37 | |
| *** EricGonczer_ has joined #openstack-infra | 13:37 | |
| *** EricGonczer_ has quit IRC | 13:38 | |
| *** priteau has quit IRC | 13:38 | |
| *** martinkopec has joined #openstack-infra | 13:39 | |
| *** gordc has quit IRC | 13:39 | |
| *** EricGonczer_ has joined #openstack-infra | 13:41 | |
| *** EricGonczer_ has quit IRC | 13:41 | |
| *** lamt has quit IRC | 13:42 | |
| *** EricGonczer_ has joined #openstack-infra | 13:42 | |
| *** Hal has quit IRC | 13:43 | |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Allow using webapp from connections https://review.openstack.org/439831 | 13:44 |
| *** Swami has joined #openstack-infra | 13:45 | |
| *** eharney has joined #openstack-infra | 13:46 | |
| openstackgerrit | Merged openstack-infra/shade master: Use requests-mock for the volume backup tests https://review.openstack.org/459898 | 13:48 |
| *** jamesmca_ has joined #openstack-infra | 13:49 | |
| openstackgerrit | Merged openstack-infra/shade master: Remove neutronclient mocks from router tests https://review.openstack.org/459872 | 13:49 |
| pabelanger | sshnaidm: where do you see that? Do you have a log file? | 13:49 |
| sshnaidm | pabelanger, I have repo files | 13:49 |
| sshnaidm | pabelanger, http://logs.openstack.org/86/459786/1/check-tripleo/gate-tripleo-ci-centos-7-ovb-ha-oooq-nv/94e1812/logs/undercloud/etc/yum.repos.d/ | 13:50 |
| *** chem has quit IRC | 13:50 | |
| *** felipemonteiro has joined #openstack-infra | 13:50 | |
| pabelanger | sshnaidm: http://git.openstack.org/cgit/openstack-infra/project-config/tree/nodepool/scripts/configure_mirror.sh are the only files we update | 13:51 |
| pabelanger | we'd have to add support to update ceph | 13:51 |
| *** EricGonc_ has joined #openstack-infra | 13:51 | |
| *** EricGonczer_ has quit IRC | 13:53 | |
| *** felipemonteiro__ has joined #openstack-infra | 13:53 | |
| mordred | pabelanger: heya - I had to run shortly after infra meeting yesterday - did we learn anything about images or repos and is there anything I can/should do to help? | 13:54 |
| sshnaidm | pabelanger, then where is this file from? http://logs.openstack.org/86/459786/1/check-tripleo/gate-tripleo-ci-centos-7-ovb-ha-oooq-nv/94e1812/logs/undercloud/etc/yum.repos.d/CentOS-QEMU-EV.repo.txt.gz | 13:54 |
| mordred | pabelanger: ooh! I tried a completely different google search than what I normally do and found this: https://www.citrix.com/blogs/2016/09/08/generating-images-for-xenserver-in-openstack/ | 13:55 |
| *** rhallisey has quit IRC | 13:55 | |
| pabelanger | mordred: I didn't get far on qemu-img for rackspace, was going to attempt again today | 13:56 |
| *** rhallisey has joined #openstack-infra | 13:56 | |
| pabelanger | mordred: interesting, will try it today | 13:57 |
| *** felipemonteiro has quit IRC | 13:57 | |
| *** vikrant has quit IRC | 13:57 | |
| mordred | pabelanger: I'm not 100% sure it's going to help us - the "convert from qcow2" steps reference the "Generate Images from VHD images" steps after the qemu-img steps | 13:58 |
| *** chlong has joined #openstack-infra | 13:58 | |
| mordred | so i'm not sure _which_ of the steps are needed - but maybe if we just do the tar step it'll be enough? | 13:58 |
| pabelanger | sshnaidm: not sure, are you installing centos-release-qemu-ev? | 13:58 |
| mordred | BobBall: ^^ any chance you're around and know more about this? | 13:58 |
| *** armax has joined #openstack-infra | 13:58 | |
| openstackgerrit | Merged openstack-infra/shade master: Replace neutronclient with REST API calls in subnet commands https://review.openstack.org/459653 | 13:58 |
| mordred | BobBall: (the goal here is being able to create working vhd images without needing our weird hacked vhd-util program - the blog post there shows a process using qemu-img to make a vhd - but I'm not sure if the vhd-util steps it lists are still needed) | 13:59 |
| pabelanger | mordred: sure, we can give tar step a go | 13:59 |
| *** dtantsur|brb is now known as dtantsur | 14:00 | |
| sshnaidm | pabelanger, nope, I suspect either nodepool or zuul do | 14:00 |
| pabelanger | sshnaidm: http://nb01.openstack.org/dib.centos-7.log | 14:02 |
| pabelanger | centos-release-openstack-ocata seems to pull it in | 14:02 |
| mordred | pabelanger: if you get a sec, mind +A'ing this: https://review.openstack.org/#/c/459109/ ? we're wasting build nodes on that job with every shade patch | 14:02 |
| pabelanger | in fact, I'm not sure we need to do that much longer for DIB | 14:02 |
| pabelanger | need to confirm with ianw if anything in devstack needs that, or we can move that step into devstack instead | 14:03 |
| *** rbrndt has joined #openstack-infra | 14:03 | |
| openstackgerrit | Matt Riedemann proposed openstack-infra/devstack-gate master: DNM: test devstack-plugin-ceph tempest_conf.sh https://review.openstack.org/459831 | 14:03 |
| sshnaidm | pabelanger, great, and centos-release-ceph-jewel too | 14:04 |
| *** lamt has joined #openstack-infra | 14:04 | |
| sshnaidm | pabelanger, it contains "mirror.centos" in it too, which overrides our repo with afs mirrors | 14:04 |
| *** jascott1 has joined #openstack-infra | 14:06 | |
| *** amotoki has quit IRC | 14:07 | |
| *** adrian_otto has joined #openstack-infra | 14:08 | |
| clarkb | pabelanger: ianw I think the iptables code is from at least lucid and maybe it only v4'd under rules? then in newer versions we symlinked? | 14:10 |
| *** Goneri has quit IRC | 14:11 | |
| *** psachin has quit IRC | 14:11 | |
| *** jascott1 has quit IRC | 14:11 | |
| BobBall | mordred: So umm the issue is that the XenServer format is slightly weird. We add some extensions to VHD to improve performance. XenServer can read VHD images created by other tools, but unfortunately some of the common operations (e.g. snapshots) actually depend on the additional metadata we include in the VHD format. | 14:11 |
| BobBall | mordred: VHDs include metadata that defines the originator; vhd-util uses "tap" as a XenServer-specific originator. If the originator is not "tap" then vhd-util refuses to operate on the VHD. | 14:12 |
| *** adrian_otto has quit IRC | 14:12 | |
| BobBall | mordred: This means that - unfortunately - you either need to use vhd-util to generate the VHD or use the vdi-import function in XenServer to import an existing VHD (from a third party) and get it re-written with the additional metadata. | 14:13 |
| openstackgerrit | Merged openstack-infra/project-config master: Move magnum devstack job for shade to experimental https://review.openstack.org/459109 | 14:14 |
| BobBall | mordred: The magic step in the blog you linked to is that it does a vdi-import which creates a VHD with the 'right' originator and the improved metadata, then grabs that directly from the disk | 14:14 |
| BobBall | mordred: I'm sure that having a custom vhd-util is preferred to having a XenServer host hanging around to do that VHD conversion :/ | 14:15 |
| BobBall | mordred: FWIW we will be updating os-xenapi to perform an automatic vdi-import if the originator is not 'tap' in the glance plugin "soon" - but there's no guarantee when Rackspace would pick up that change | 14:16 |
| openstackgerrit | David Moreau Simard proposed openstack-infra/tripleo-ci master: Do not merge: Test Ceph Jewel 10.2.7 https://review.openstack.org/460121 | 14:17 |
| pabelanger | clarkb: look at history for puppet-iptables, that might line up. However, it appears nothing today uses it | 14:17 |
| BobBall | Sorry :) | 14:17 |
| pabelanger | clarkb: atleast on ubuntu | 14:17 |
| clarkb | pabelanger: ya pretty sure at this point your change to fix is fine | 14:18 |
| *** prateek has quit IRC | 14:18 | |
| clarkb | BobBall: out of curiousity why not call it something else then get the image format supproted by qemu-img? as is its just straight up confusing | 14:18 |
| pabelanger | sshnaidm: k, I'll likley test this week devstack without centos repos first installed, see how it functions. Then, update devstack if needed | 14:19 |
| BobBall | clarkb: The original intention was to be compatible with all VHDs. I don't understand why vhd-util is refusing to co-operate with other VHDs. IMO that should be the real fix. Unfortunately as there's a very easy workaround (using vdi-import) it's been a struggle to get a fix on the roadmap. I will try again. | 14:20 |
| *** amotoki has joined #openstack-infra | 14:21 | |
| *** prateek has joined #openstack-infra | 14:25 | |
| *** prateek_ has joined #openstack-infra | 14:27 | |
| *** gordc has joined #openstack-infra | 14:27 | |
| *** pradk has joined #openstack-infra | 14:27 | |
| *** Goneri has joined #openstack-infra | 14:29 | |
| *** prateek has quit IRC | 14:29 | |
| *** srwilkers has quit IRC | 14:29 | |
| *** tphummel has joined #openstack-infra | 14:30 | |
| *** mat128 has joined #openstack-infra | 14:30 | |
| *** felipemonteiro__ has quit IRC | 14:31 | |
| *** felipemonteiro has joined #openstack-infra | 14:31 | |
| *** prateek_ has quit IRC | 14:31 | |
| *** prateek_ has joined #openstack-infra | 14:32 | |
| *** ZZelle has joined #openstack-infra | 14:32 | |
| *** ZZelle is now known as Guest22141 | 14:33 | |
| clarkb | BobBall: right I guess from our perspective its better for us if we can generate images that will work independent of whatever $cloudprovider does | 14:34 |
| mordred | BobBall: cool - well, thanks for the information, I at least understand what's going on now :) | 14:34 |
| mtreinish | clarkb, mordred, pabelanger: if you get a sec: https://review.openstack.org/#/c/459865/2 should hopefully fix the subunit-workers so we can start populating subunit2sql again | 14:34 |
| clarkb | since wecan't control them. So having a xenserver specific image format that qemu can generate for us is betterthan cloud backend vhd-util accepting vanilla vhds (though thats good too and would eventually lead to happier clouds I thinl) | 14:35 |
| mordred | BobBall: vhd-util itself works pretty well - the biggest problem we have with _it_right now is that there isn't a version that's apt-get installable on ubuntu that works | 14:35 |
| mordred | BobBall: we have to carry this patch: https://github.com/emonty/vhd-util/blob/master/debian/patches/citrix which I found on the internet :) | 14:36 |
| mordred | apparenlty it's not a good patch, so ubuntu/debian aren't interested in carrying it themselves, and apparently nobody really got it moving upstream or something | 14:36 |
| *** erikmwil_ has joined #openstack-infra | 14:36 | |
| *** baoli has quit IRC | 14:36 | |
| mordred | I understand so little of the actual goings on that I'm not much help in trying to drive it to a real resolution | 14:36 |
| *** erikmwilson has quit IRC | 14:37 | |
| *** amotoki has quit IRC | 14:38 | |
| *** baoli has joined #openstack-infra | 14:38 | |
| *** salv-orlando has joined #openstack-infra | 14:38 | |
| *** martinkopec has quit IRC | 14:38 | |
| mordred | BobBall: tbh - if there was a citrix apt repo with a vhd-util in it that worked for xenserver images without having to install the rest of xenserver, that would also be a completely lovely solution from my pov - I don't mind needing to install the tool- I mostly am concerned with maintaining a fork I don't actually understand | 14:38 |
| *** Guest22141 has quit IRC | 14:39 | |
| *** srwilkers has joined #openstack-infra | 14:39 | |
| *** salv-orl_ has joined #openstack-infra | 14:39 | |
| mordred | BobBall: which is all me babbling about things that would make me happy if someone else did them - but I'm also more than happy/willing to do some work myself to help if I knew what to do or who to bother or whatnot | 14:40 |
| mordred | BobBall: (also, super excited about the update to the glance plugin - I think that's exactly how that should work in openstack | 14:41 |
| mordred | BobBall: like, I've been arguing that OpenStack should _always_ accept raw and qcow2 images, even if it's not the ideal format for a given cloud, and be able to convert them to the native format needed/preferred by the cloud | 14:42 |
| BobBall | mordred: The frustrating part is the vhd-util patch is that it's adding a whole bunch of functionality (convert) that isn't in our vhd-util... It's good stuff IMO but getting it included in tree means people want to verify + qa it ... | 14:42 |
| mordred | so sounds like that's at leat a step forward | 14:42 |
| mordred | BobBall: yuck | 14:42 |
| mordred | BobBall: fwiw, using that copy of vhd-util to produce a working image is a fascinatingly opaque set of two steps ... | 14:43 |
| *** salv-orlando has quit IRC | 14:43 | |
| BobBall | mordred: I hope qcow2 support in XenServer would be the best way forward. That depends on XS using upstream qemu. | 14:43 |
| *** amotoki has joined #openstack-infra | 14:43 | |
| mordred | https://github.com/openstack/diskimage-builder/blob/347833856ee40dfad235de54eebf9d9853381116/diskimage_builder/lib/img-functions#L145-L151 | 14:43 |
| *** prateek_ has quit IRC | 14:44 | |
| mordred | BobBall: yah- qcow2 support in XenServer would be amazing | 14:44 |
| BobBall | mordred: but our driver fix would be the best short-term workaround. | 14:44 |
| clarkb | fwiw I'd happily have the convert stuff live in qemu-img :) | 14:44 |
| mordred | yah | 14:44 |
| pabelanger | ++ | 14:44 |
| mordred | BobBall: is it just the metadata tap field that needs done? or are there actual structural things about the vhd produced by qemu-img that are also incorect for xenserver? | 14:45 |
| pabelanger | cmurphy: so, looks like we just need to remove puppet::jenkinsuser now. | 14:46 |
| BobBall | mordred: XenServer has some additional metadata to spead up reads when whole blocks haven't been written to. Originally weren't meant to depend on this metadata existing, but it seems that a reliance on the originator crept in - demonstrating we didn't have any testing on non-XenServer extended VHDs | 14:46 |
| *** felipemonteiro__ has joined #openstack-infra | 14:46 | |
| cmurphy | pabelanger: yep! | 14:46 |
| *** marst has quit IRC | 14:47 | |
| clarkb | mtreinish: fungi I've just approved that change lets see if that fixes everything | 14:47 |
| fungi | coolness | 14:47 |
| cmurphy | and the puppetlabs repos in the template class | 14:47 |
| mordred | BobBall: nod - to noting the tap in the header winds up being a shorthand check to make sure the metadata is there | 14:47 |
| *** dave-mccowan has joined #openstack-infra | 14:47 | |
| mordred | BobBall: but there is actually metadata that xenserver winds up relying on, so it's a legit check | 14:47 |
| BobBall | mordred: I can't promise that modifying the originator would make it work - but it might be worth a quick qemu hack to check if it does. | 14:47 |
| *** dhinesh has joined #openstack-infra | 14:48 | |
| openstackgerrit | Rosario Di Somma proposed openstack-infra/shade master: Add a _normalize_volume_backups method https://review.openstack.org/460127 | 14:48 |
| pabelanger | cmurphy: I am thinking, maybe we create a jenkins-slave element in project-config, port so of the logic into that, and 3rd party CI will still use that element, if they still run against a jenkins master | 14:49 |
| mtreinish | clarkb: hopefully it will | 14:49 |
| BobBall | mordred: if qemu-img convert actually zeros the un-used portion of the header (which I assume it does) then we might get lucky. | 14:49 |
| cmurphy | pabelanger: sounds good | 14:49 |
| pabelanger | k, I'll start on that now | 14:49 |
| cmurphy | pabelanger: i don't have 100% understanding on what the jenkins user is for nowadays :) | 14:49 |
| *** shu-mutou-AWAY has quit IRC | 14:49 | |
| clarkb | cmurphy: its the user that zuul logs into the test nodes as | 14:50 |
| cmurphy | clarkb: isn't there a zuul user? | 14:50 |
| clarkb | we also have a zuul user on the test nodes but haven't finished converted all the things to use it (mirror setup, initial ssh login, etc) | 14:50 |
| pabelanger | cmurphy: ya, for now, just a user account. We could actually switch to the zuul user but not sure if we want to do that along side zuulv3-dev.o.o | 14:50 |
| *** felipemonteiro has quit IRC | 14:50 | |
| pabelanger | clarkb: zuul user should work now | 14:50 |
| pabelanger | we'd just have to update nodepool.o.o to start using it | 14:51 |
| cmurphy | lol there will probably be a lot of side effects to trying to change it | 14:51 |
| pabelanger | ya | 14:51 |
| clarkb | ah ok | 14:51 |
| pabelanger | I don't mind keeping jenkins user a little longer | 14:51 |
| clarkb | pabelanger: I don't recall zuul sudo things? | 14:51 |
| clarkb | we just touched all that code and it was all jenkins specific iirc | 14:51 |
| clarkb | so there are likely some remaining things that need to be zuulified before we can switch? | 14:52 |
| pabelanger | ya, lets hold off for now | 14:52 |
| pabelanger | at least until puppet is all done :) | 14:52 |
| *** martinkopec has joined #openstack-infra | 14:53 | |
| *** tonytan_brb has joined #openstack-infra | 14:53 | |
| *** amotoki has quit IRC | 14:53 | |
| clarkb | hogepodge: dims fyi http://status.openstack.org/elastic-recheck/#1660973 is affecting gate-k8s-cloud-provider-golang-dsvm-conformance-ubuntu-xenial | 14:54 |
| clarkb | hogepodge: dims I think that means the job is misconfigured and isn't starting a placement api? | 14:54 |
| BobBall | mordred: At first quick read of the code I believe that we do correctly handle the batmap not existing irrespective of the originator type (code at https://github.com/xapi-project/blktap/blob/master/vhd/lib/libvhd.c if you care). This means that if you fudge the originator you might get lucky. This wouldn't work in the general case, but might work for openstack, because OS always checkpoints VDIs as soon as they are imported to allow caching of the | 14:54 |
| *** marst has joined #openstack-infra | 14:54 | |
| dims | hogepodge : not doing anything special there, just plain devstack with some hooks to start k8s | 14:55 |
| *** tonytan4ever has quit IRC | 14:55 | |
| dims | works most of the time... | 14:55 |
| dims | clarkb : since i am the only one affected, did not complain :) | 14:55 |
| *** dhinesh has quit IRC | 14:55 | |
| *** marst_ has joined #openstack-infra | 14:55 | |
| *** marst has quit IRC | 14:55 | |
| clarkb | dims: ok, just noticed it in elastic-recheck as our top issue and its largely only affecting that job | 14:55 |
| clarkb | http://logs.openstack.org/61/456561/22/check/gate-k8s-cloud-provider-golang-dsvm-conformance-ubuntu-xenial/e824bed/logs/screen-n-cpu.txt is an example | 14:56 |
| mordred | BobBall: it looks like creator app is actually in the footer? | 14:56 |
| *** dhinesh has joined #openstack-infra | 14:56 | |
| dims | :) | 14:56 |
| openstackgerrit | Merged openstack-infra/system-config master: Correct the ca-cert for mqtt on subunit worker https://review.openstack.org/459865 | 14:56 |
| clarkb | dims: I don't see placement in enabled services? | 14:56 |
| BobBall | mordred: Header and footer is the same. In VHD the header is a "backup" of the footer - but actually the "backup" is more reliable when you consider resizing VHDs where the footer might be temporarily not there. | 14:56 |
| *** xarses_ has joined #openstack-infra | 14:57 | |
| mordred | gotcha. so when I'm looking at code in qemu for creating the footer struct, that's actually the thing that's doing the work | 14:57 |
| *** dhinesh has quit IRC | 14:57 | |
| clarkb | dims: that job has an overridden services list. | 14:58 |
| mordred | BobBall: while I'm learning things - there seem to be "dynamic" and "fixed" types of images - if I'm just converting an existing raw, does it matter which of those we use? | 14:58 |
| BobBall | mordred: I think we always use dynamic | 14:58 |
| *** isaacb has quit IRC | 14:59 | |
| *** andreww has joined #openstack-infra | 14:59 | |
| mordred | kk. cool | 14:59 |
| mordred | thanks | 14:59 |
| BobBall | mordred: IIRC fixed images don't allow chains of VHDs - so no snapshots. | 14:59 |
| clarkb | dims: nodepool jobs have gone the route of disabling services in default devstack that it doesn't want https://git.openstack.org/cgit/openstack-infra/project-config/tree/jenkins/jobs/nodepool.yaml#n24 so that things generally work as new required services are added, maybe want to convert to that in the golang job? | 14:59 |
| mordred | BobBall: yah- but that makes sense for us since we're not actually coming in with a VHD that was produced in a more elegant manner anyway | 14:59 |
| mordred | oh - wait - you're saying ... nevermind | 14:59 |
| mordred | I can read english now | 15:00 |
| *** tongli has quit IRC | 15:00 | |
| *** ihrachys has quit IRC | 15:00 | |
| BobBall | :) Got a meeting now. Back later. | 15:00 |
| dims | clarkb : good point, will make the change | 15:00 |
| *** ihrachys has joined #openstack-infra | 15:00 | |
| *** rhallisey has quit IRC | 15:01 | |
| mordred | BobBall: thank you so much for your help! | 15:01 |
| *** tjones has joined #openstack-infra | 15:01 | |
| *** xarses_ has quit IRC | 15:01 | |
| *** jcoufal_ has joined #openstack-infra | 15:01 | |
| clarkb | dims: sorry to bother, I have just been checking elastic-recheck every morning and have tried to make it a thing to track down the top bug :) | 15:01 |
| *** xarses_ has joined #openstack-infra | 15:02 | |
| clarkb | though our categorization rate is bad right now so maybe I shoudl spend more time categorizing | 15:02 |
| dims | clarkb : not a bother at all :) thanks for doing that! | 15:02 |
| mtreinish | clarkb: heh, the failure rate is low, it makes keeping the percentage high difficult | 15:03 |
| *** andreww has quit IRC | 15:03 | |
| *** lin_yang has joined #openstack-infra | 15:04 | |
| *** rossella_s has quit IRC | 15:04 | |
| *** jcoufal has quit IRC | 15:04 | |
| *** andreww has joined #openstack-infra | 15:04 | |
| fungi | that's certainly a nice problem, for a change ;) | 15:04 |
| clarkb | mtreinish: ya that too. Is http://logs.openstack.org/24/456824/7/check/gate-tempest-dsvm-neutron-full-ubuntu-xenial/92cf8a4/console.html.gz#_2017-04-21_06_00_22_504199 a tempest bug? it sets container format and disk format to null then it goes to PUT the image and fails | 15:05 |
| *** rossella_s has joined #openstack-infra | 15:05 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/project-config master: Add jenkins-slave element for nodepool https://review.openstack.org/460137 | 15:06 |
| clarkb | http://logs.openstack.org/96/457696/1/gate/gate-tempest-dsvm-neutron-full-ubuntu-xenial/71cc014/console.html.gz#_2017-04-18_17_39_36_163783 thats a fun one. Devstack took an hour to run then the rest of the job timed out | 15:06 |
| *** rhallisey has joined #openstack-infra | 15:06 | |
| *** wolverineav has joined #openstack-infra | 15:06 | |
| pabelanger | clarkb: cmurphy: before I start testing, what do you think about embedding our SSH key for jenkins user? ^ | 15:06 |
| pabelanger | clarkb: cmurphy: open to suggestions too | 15:06 |
| clarkb | but ^ is easy to make a query for so I'll try to get that up soon | 15:06 |
| *** xarses_ has quit IRC | 15:06 | |
| cmurphy | pabelanger: I'm a fan of simple solutions | 15:07 |
| mtreinish | clarkb: it's likely a configuration bug: https://github.com/openstack/tempest/blob/master/tempest/api/volume/test_volumes_negative.py#L40-L57 | 15:07 |
| *** xarses_ has joined #openstack-infra | 15:07 | |
| mtreinish | it pulls in the container and disk format from config | 15:07 |
| cmurphy | pabelanger: but not sure if you want that key on 3rd party images? | 15:07 |
| mtreinish | although I don't remember those list opts being a thing | 15:07 |
| * mtreinish checks | 15:07 | |
| mtreinish | heh, they default to all the things: https://github.com/openstack/tempest/blob/master/tempest/config.py#L506-L514 | 15:08 |
| *** rhallisey has quit IRC | 15:08 | |
| *** andreww has quit IRC | 15:08 | |
| *** rhallisey has joined #openstack-infra | 15:08 | |
| *** jascott1 has joined #openstack-infra | 15:08 | |
| pabelanger | cmurphy: ya, 3rd party CI today set up the NODEPOOL_SSH_KEY variable with their own key, otherwise we default to ours. This is how prepare-node works right now | 15:09 |
| pabelanger | so, this should be a noop | 15:09 |
| cmurphy | pabelanger: oh i see the :- | 15:09 |
| clarkb | pabelanger: left some comments in the chaneg | 15:09 |
| clarkb | mtreinish: what does a disk_format default of all the things mean :) | 15:10 |
| openstackgerrit | Davanum Srinivas (dims) proposed openstack-infra/project-config master: Use disable_service for golang dsvm jobs https://review.openstack.org/460139 | 15:10 |
| dims | clarkb : ^ | 15:10 |
| mtreinish | clarkb: it's a list opt telling tempest which disk formats are allowed on glance. The default is all the formats glance could support | 15:10 |
| clarkb | mtreinish: gotcha | 15:10 |
| clarkb | mtreinish: so something has to choose the format for a particular image then? | 15:11 |
| *** amotoki has joined #openstack-infra | 15:11 | |
| mtreinish | clarkb: I think it's a glance bug actually, the nulls are in the response | 15:11 |
| *** rhallisey has quit IRC | 15:11 | |
| mtreinish | although tempest isn't specifying anything in the request body | 15:11 |
| clarkb | dims: +2 | 15:12 |
| *** isaacb has joined #openstack-infra | 15:12 | |
| clarkb | mtreinish: I think thats because glance is helpful and lets you tell it what those things are later if you are lazy? | 15:12 |
| pabelanger | clarkb: left reply on how to move forward | 15:12 |
| clarkb | just not so late that the image upload is happening | 15:12 |
| mtreinish | clarkb: well there is this: http://logs.openstack.org/24/456824/7/check/gate-tempest-dsvm-neutron-full-ubuntu-xenial/92cf8a4/logs/screen-g-api.txt.gz?level=TRACE#_2017-04-21_05_50_23_009 | 15:13 |
| *** chem has joined #openstack-infra | 15:13 | |
| clarkb | mtreinish: thats to prevent skipping in line | 15:13 |
| *** jascott1 has quit IRC | 15:13 | |
| mtreinish | but that's on the failed PUT (which is the image upload) | 15:13 |
| openstackgerrit | Paul Belanger proposed openstack-infra/project-config master: Add jenkins-slave element for nodepool https://review.openstack.org/460137 | 15:14 |
| clarkb | so possibly the error bubbled out of the api is not correct and the logs show the correct issue is in glance state transitions? | 15:14 |
| pabelanger | okay, I am going to create an image using ^ | 15:14 |
| *** rhallisey has joined #openstack-infra | 15:15 | |
| clarkb | pabelanger: ++ to assuming key type is part of key value | 15:15 |
| pabelanger | great! We'll have to get 3rd party CI operators on board too. Likely create a new variable and deprecate the current | 15:16 |
| clarkb | mordred: re http://logs.openstack.org/68/364368/5/gate/gate-tempest-dsvm-neutron-full-ubuntu-xenial/b5a7146/console.html.gz#_2017-04-17_03_33_39_970946 notice how the pip collecting foo messages all seem happy then its easy_install for pbr that breaks? What do you think the chances are that easy_install http < pip http? | 15:18 |
| mtreinish | clarkb: so that failure is a tempest bug, but from an earlier rev in the patch (which was broken) | 15:18 |
| clarkb | mordred: taking that further do you think it would make sense to preinstall pbr with pip as the first thing we do? | 15:18 |
| mtreinish | clarkb: https://review.openstack.org/#/c/456824/7..9/tempest/api/volume/test_volumes_negative.py | 15:18 |
| mtreinish | clarkb: this is why I only care about failures in gate :) | 15:18 |
| mtreinish | (rev 9 there is what ended up landing) | 15:18 |
| clarkb | mtreinish: I'm looking at the uncategorized gate fails list | 15:19 |
| mordred | clarkb, pabelanger, BobBall: http://paste.openstack.org/show/608085/ I think this is a patch to qemu-img that would implement what BobBall was tlaking about | 15:19 |
| clarkb | but that job is in check | 15:19 |
| mordred | clarkb: looking | 15:19 |
| mtreinish | clarkb: really? It should filter on gate for the uncategorized list | 15:19 |
| openstackgerrit | Doug Hellmann proposed openstack-infra/project-config master: import downpour https://review.openstack.org/460098 | 15:19 |
| pabelanger | mordred: woah, nice | 15:19 |
| *** vsaienk0 has quit IRC | 15:19 | |
| mtreinish | clarkb: oh, I bet logstash screwed up the metadata again | 15:20 |
| mordred | clarkb: this is in our gate-specific venv right? | 15:20 |
| clarkb | mordred: ya its where devstack-gate installs the things it needs from python | 15:20 |
| clarkb | so shouldn't step on devstack (which amy fail later I guess) | 15:20 |
| mordred | clarkb: yah- I think us pre-installing pbr with pip in that case is copletely fine | 15:21 |
| clarkb | mordred: do you think it will make things more reliable though? just seems like pips ability to handle http and pypi stuff is more solid based on that | 15:22 |
| openstackgerrit | Thomas Morin proposed openstack-infra/project-config master: n8g-bagpipe: use pre/post gate hooks to fullstack job https://review.openstack.org/460168 | 15:23 |
| *** beslemon has joined #openstack-infra | 15:24 | |
| openstackgerrit | Martin André proposed openstack-infra/tripleo-ci master: [WIP] add containerized deployment on multinode https://review.openstack.org/454152 | 15:27 |
| mtreinish | fungi, clarkb: btw, the cert patch landed ~30min ago (hopefully enough time for puppet) if you wanted to try round 3 of getting the worker running :) | 15:28 |
| clarkb | mtreinish: I just restarted the worker | 15:29 |
| clarkb | the yaml conf file had the new cert path in it | 15:29 |
| *** bswartz has quit IRC | 15:29 | |
| mtreinish | clarkb: and the cert file was created too? | 15:30 |
| mtreinish | clarkb: :/ I'm still not seeing anything on mqtt or the sql | 15:30 |
| clarkb | its still getting econnreset | 15:30 |
| *** fandi has joined #openstack-infra | 15:30 | |
| clarkb | are you sure that sslv23 issue on the server side isn't at least partially to blame? | 15:30 |
| mordred | clarkb: yes, I do think it will make things more reliable | 15:31 |
| clarkb | I think you need to be using tlsv1 or greater | 15:31 |
| *** yolanda has quit IRC | 15:31 | |
| clarkb | because openssl just won't have sslv2 or v3 compiled in anymore so anything trying to negotiate that is sol | 15:31 |
| mtreinish | clarkb: I just connected using: https://docs.openstack.org/infra/system-config/firehose.html#using-ssl-tls | 15:31 |
| *** EricGonc_ has quit IRC | 15:31 | |
| mtreinish | using the same ca cert from: https://support.comodo.com/index.php?/Knowledgebase/Article/View/921/75/new-instantssl-ca-bundle | 15:32 |
| mtreinish | and that python example script which should be the same | 15:32 |
| strigazi | hi clarkb, in our tests (magnum) we spawn vms that need to talk to several OS service endpoints (keystone, heat, magnum). Recently, we started getting 'No route to host' error when contacting the keystone api from inside the vms. | 15:32 |
| *** Hal has joined #openstack-infra | 15:34 | |
| strigazi | clarkb, we don't see this error in ocata. Any ideas? | 15:34 |
| mtreinish | clarkb: it might be the ssl version thing, but can you check the mosquitto logs for my connection info to see what it says | 15:35 |
| openstackgerrit | Thomas Morin proposed openstack-infra/project-config master: n8g-bagpipe: use pre/post gate hooks to fullstack job https://review.openstack.org/460168 | 15:35 |
| clarkb | mordred: it bugs me that your array isn't big neough to be null terminated in that diff (for "qemu") | 15:35 |
| clarkb | mordred: and I'm sure someone will grump about not using strncpy? | 15:35 |
| clarkb | mtreinish: ya give me a sec | 15:36 |
| *** chem has quit IRC | 15:37 | |
| *** vsaienk0 has joined #openstack-infra | 15:37 | |
| *** salv-orl_ has quit IRC | 15:37 | |
| mordred | clarkb: sorry - my pure-C is rusty after all the drizzle c++ | 15:38 |
| dhellmann | does anyone know/remember if the archives from the old launchpad-driven mailing list were imported into mailman when we moved to that? | 15:38 |
| clarkb | mtreinish: assuming your IP addr last octet is 51 it doesn't say anything about ssl | 15:39 |
| *** hamzy has quit IRC | 15:39 | |
| mtreinish | clarkb: yeah that should be me | 15:39 |
| clarkb | strigazi: recently as in the last day? there was a slight misconfiguration for iptables that should be corrected | 15:39 |
| clarkb | strigazi: if jobs run now do you still see it? | 15:40 |
| mtreinish | clarkb: does it say 8883 on it though? that listener is only ssl traffic | 15:40 |
| mtreinish | (that's also the port I set on my clients) | 15:40 |
| clarkb | mordred: on further reading it looks like old uses of qemu were actually not null terminated. The memcpy on line 36 | 15:41 |
| mordred | clarkb: oh- actually - this is all because I was working backwards from the memcpylater | 15:41 |
| mordred | yah | 15:41 |
| *** Sukhdev_ has joined #openstack-infra | 15:41 | |
| clarkb | mordred: the real bug is that you are setting it to "tapu" | 15:41 |
| strigazi | clarkb I will try again. I was on holidays, so I noticed yesterday | 15:41 |
| mordred | clarkb: I'm not - I memset 0 | 15:41 |
| mtreinish | clarkb: what's the host os on subunit-worker01.o.o? | 15:41 |
| clarkb | mordred: oh you do cool | 15:41 |
| clarkb | mtreinish: precise | 15:42 |
| clarkb | mtreinish: oh that may be it :) | 15:42 |
| mtreinish | that might have something to do with it :) | 15:42 |
| clarkb | mtreinish: let me see if 02 is working | 15:42 |
| *** jpena is now known as jpena|brb | 15:42 | |
| *** jordanP has quit IRC | 15:42 | |
| *** jordanP has joined #openstack-infra | 15:42 | |
| mordred | clarkb: of course, now I don't know if the non-null-terminated field wants to be "tap\0" or "\0tap" or " tap" | 15:42 |
| *** prateek_ has joined #openstack-infra | 15:42 | |
| clarkb | mordred: ya this got complicated fast :) | 15:43 |
| clarkb | mordred: slightly grump that qemu doesn't null terminate :/ | 15:43 |
| clarkb | then its easy | 15:43 |
| mordred | clarkb: https://github.com/xapi-project/blktap/blob/master/vhd/lib/libvhd.c#L277 | 15:43 |
| mordred | it wants to be "tap\0" | 15:43 |
| openstackgerrit | Saad Zaher proposed openstack-infra/project-config master: Added api-ref jobs for freezer-api https://review.openstack.org/460179 | 15:43 |
| mordred | so the memset 0 will do the right thing | 15:43 |
| openstackgerrit | Sagi Shnaidman proposed openstack-infra/tripleo-ci master: DONT REVIEW: testing oooq https://review.openstack.org/459766 | 15:44 |
| dhellmann | https://answers.launchpad.net/launchpad/+question/238317 implies that the archives were downloaded, but http://lists.openstack.org/pipermail/openstack/2013-July/thread.html doesn't seem to have the old messages | 15:44 |
| *** annegentle has joined #openstack-infra | 15:44 | |
| clarkb | mordred: also will compiler overflow that array? Generally if you array = "foo" its magical null termination. The old code just memset into the dest struct directly and didn't deal with that | 15:44 |
| clarkb | mordred: my C is rusty too :) | 15:45 |
| dhellmann | nor does http://lists.openstack.org/pipermail/openstack-dev/ | 15:45 |
| *** Rockyg has joined #openstack-infra | 15:45 | |
| clarkb | dhellmann: -dev goes back an extra year though | 15:45 |
| dhellmann | clarkb : yeah, I did notice that | 15:46 |
| *** kzaitsev_ws has quit IRC | 15:46 | |
| dhellmann | but the july page doesn't include some of the older items I'm finding on the LP list | 15:46 |
| clarkb | ah | 15:46 |
| dhellmann | oh, and it doesn't go all the way back to 2010, which the LP list does | 15:46 |
| clarkb | ya | 15:46 |
| dhellmann | I was looking at the newest stuff on the LP list, not oldest | 15:46 |
| mordred | clarkb: http://paste.openstack.org/show/608087/ how about that | 15:46 |
| dhellmann | I suppose I could ask the LP folks to prepare another mbox file for me, but we've done that several times already so I was hoping the results were available somewhere on our current list server | 15:47 |
| *** mdrabe has quit IRC | 15:47 | |
| fungi | dhellmann: confirmed, the general openstack ml archive on disk only goes back to 2013-August | 15:48 |
| clarkb | mordred: looks good | 15:48 |
| fungi | and openstack-dev to 2012-August | 15:48 |
| dhellmann | the operator list goes back to 2010, but not all the way to july | 15:48 |
| dhellmann | http://lists.openstack.org/pipermail/openstack-operators/ vs https://lists.launchpad.net/openstack/thrd128.html | 15:48 |
| fungi | dhellmann: there are some files in rockstar's homedir on the listserv which might have been the in-progress import | 15:49 |
| mordred | pabelanger, clarkb: great. I've got a few things on my stack before I can test that, but I'll test in just a bit | 15:49 |
| openstackgerrit | Matt Riedemann proposed openstack-infra/devstack-gate master: DNM: test devstack-plugin-ceph tempest_conf.sh https://review.openstack.org/459831 | 15:49 |
| dhellmann | fungi : oh, interesting | 15:49 |
| fungi | trying to see what the state there is | 15:49 |
| mordred | if that actuallyworks, we can work on upstreaming it- althoughwe'll STILL have an issue of "where do we put that while we wait for it to be in a place" | 15:49 |
| clarkb | mtreinish: 02 is getting the same behavior on both sides. and its trusty | 15:50 |
| clarkb | mtreinish: I think trusty should be fine with ssl | 15:50 |
| fungi | dhellmann: looks like they were last touched in july 2013 which seems about right | 15:50 |
| dhellmann | fungi: I don't know how mailman works with importing bulk data like this, but I wonder if it would be better to make a new list (openstack-old or something) and import them there, then make it impossible to subscribe but possible to look at the archives | 15:50 |
| dhellmann | or maybe that's more complicated than it needs to be | 15:50 |
| fungi | rockstar: do you recall how far you got back then (if you're around)? | 15:50 |
| openstackgerrit | Merged openstack-infra/project-config master: Use disable_service for golang dsvm jobs https://review.openstack.org/460139 | 15:51 |
| *** MasterOfBugs has joined #openstack-infra | 15:51 | |
| mtreinish | clarkb: err, then I'm not sure what's up | 15:51 |
| openstackgerrit | Ben Nemec proposed openstack-infra/tripleo-ci master: TEST: Bisect heat https://review.openstack.org/460188 | 15:52 |
| openstackgerrit | Ben Nemec proposed openstack-infra/tripleo-ci master: TEST: Bisect heat https://review.openstack.org/460189 | 15:52 |
| *** watanabe_isao has joined #openstack-infra | 15:52 | |
| dirk | pabelanger: sorry it took me some time to get back to this. here is the opensuse build log -- https://pastebin.com/raw/7cT9JZDK | 15:52 |
| clarkb | mtreinish: are there docs for the mqtt lib? I wonder if its just picking the first ssl version locally and we can override maybe? | 15:53 |
| *** gcb has joined #openstack-infra | 15:53 | |
| mtreinish | clarkb: I was just looking at that, there is a tls_version arg: https://github.com/eclipse/paho.mqtt.python/blob/master/src/paho/mqtt/client.py#L598 | 15:53 |
| mtreinish | the dict we pass in for publish_single gets passed to that func as kwargs | 15:54 |
| fungi | dhellmann: yeah, so poking around is looks like what was imported was the list settings and subscribers, not the archives | 15:54 |
| dhellmann | fungi : are the archives there? | 15:54 |
| dhellmann | for today I'd be happy with an mbox file (or set of them, or whatever) | 15:54 |
| fungi | dhellmann: no, we don't have downloads of the archives on the listserv that i'm finding, at least | 15:54 |
| dhellmann | ah, well | 15:54 |
| dirk | pabelanger: ah, ok so the failure above is because of https://review.openstack.org/#/c/447220/ missing | 15:54 |
| clarkb | mtreinish: maybe try force set tlsv1? | 15:55 |
| *** tie has quit IRC | 15:55 | |
| *** mdrabe has joined #openstack-infra | 15:55 | |
| dhellmann | fungi: I'll ask the LP folks for them again, and maybe we can figure out some way to host them ourselves | 15:55 |
| *** gcb has quit IRC | 15:55 | |
| fungi | dhellmann: yeah, i was about to start digging around on lp seeing if i could find mbox files | 15:55 |
| mtreinish | clarkb: yeah, I'll write up a patch for that. One sec | 15:55 |
| clarkb | mtreinish: also https://github.com/eclipse/paho.mqtt.python/commit/01cc0139a16c7d2a860ede3304c7be7216fd636e | 15:55 |
| pabelanger | dirk: rebuild again, we just removed puppet-iptables | 15:55 |
| pabelanger | dirk: but haven't landed suse firewall rules just yet | 15:56 |
| *** gcb has joined #openstack-infra | 15:56 | |
| *** watanabe_isao has quit IRC | 15:56 | |
| dhellmann | fungi : the service doesn't create them automatically, we have to ask. I'll do that, get the files, and then we can talk about putting them somewhere public | 15:56 |
| dhellmann | fungi : https://answers.launchpad.net/launchpad/+question/628770 | 15:57 |
| fungi | dhellmann: oh, excellent. lmk when you have them and i'll see what i can do to import somehow | 15:57 |
| dirk | pabelanger: ah, that's good. can we W+1 https://review.openstack.org/#/c/459055/ ? | 15:57 |
| *** Julien-zte has quit IRC | 15:57 | |
| fungi | dhellmann: or at least i'll find a volunteer to take it on | 15:57 |
| dirk | it shouldn't harm existing images | 15:58 |
| dhellmann | fungi : sounds good | 15:58 |
| pabelanger | dirk: do you mind rebuilding using that version of project-config? | 15:58 |
| dirk | pabelanger: sure, I alraedy have that in my local checkout otherwise I can't do anything | 15:58 |
| *** jascott1 has joined #openstack-infra | 15:58 | |
| openstackgerrit | Matthew Treinish proposed openstack-infra/puppet-subunit2sql master: Force using TLS_v1 https://review.openstack.org/460194 | 15:58 |
| pabelanger | dirk: ya, that should be enough | 15:58 |
| mtreinish | clarkb: ^^^ | 15:58 |
| pabelanger | dirk: if you use tools/build-images.sh | 15:59 |
| *** Julien-zte has joined #openstack-infra | 15:59 | |
| clarkb | speaking of iptablse on suse, is there a good way to do something like iptables-persistent and get yast out of the way? | 15:59 |
| clarkb | edtiing firewall rules on tumbleweed is interesting | 15:59 |
| pabelanger | I am not sure, I haven't build opensuse yet | 16:00 |
| openstackgerrit | Matthew Treinish proposed openstack-infra/puppet-subunit2sql master: Force using TLS_v1 https://review.openstack.org/460194 | 16:00 |
| *** prateek_ has quit IRC | 16:00 | |
| pabelanger | I plan on working on xenial nodepool-builder later today too | 16:00 |
| dirk | clarkb: as I'm employed by suse, let's say I have nothing to say about that ;-) | 16:00 |
| pabelanger | but, that all depends on our vhd-utils package :) | 16:00 |
| dirk | clarkb: as far as I know the custom rules in susefirewall2 , which the review above implements, is the best "proper" way | 16:00 |
| *** ZZelle has joined #openstack-infra | 16:00 | |
| *** beslemon has quit IRC | 16:00 | |
| dirk | everyone I know of just has a locally hacked script for persisting iptables rules | 16:01 |
| *** ZZelle is now known as Guest14948 | 16:01 | |
| fungi | dhellmann: https://answers.launchpad.net/launchpad/+question/161282 seems to confirm (i guess you found that one as well) | 16:01 |
| dirk | I think iptables-persistent would be a good thing to have, I'll just need to find someone who has the cycles to submit and maintain it and get it included | 16:01 |
| *** jlk has quit IRC | 16:01 | |
| clarkb | dirk: there isn't an equivalent of iptables-persistent? fwiw we explicitly disable and kill the firewall manager daemon on centos/fedora they comes by default and we don't install ubuntus | 16:01 |
| *** beslemon has joined #openstack-infra | 16:02 | |
| clarkb | dirk: we initially did that because firewalld made libvirt run very slowly | 16:02 |
| clarkb | nto sure if its still an issue or not | 16:02 |
| *** jlk has joined #openstack-infra | 16:02 | |
| dhellmann | fungi : yeah, that's what led me to open a question. When I started it, I found 4-5 other older questions for the same archives, which is what led me here to ask if anyone remembered what happened to that data. :-) | 16:02 |
| *** jlk has quit IRC | 16:02 | |
| *** jlk has joined #openstack-infra | 16:02 | |
| fungi | heh | 16:02 |
| clarkb | dirk: since this is entirely new distro image builds I'm less worried about it ther ebut thats some of the history behind why we ended up with iptables-persistent everywhere | 16:03 |
| *** andreas_s has quit IRC | 16:03 | |
| fungi | dhellmann: oh! since reed was the one to file that earlier question, i checked his homedir and found it | 16:03 |
| dhellmann | ah! | 16:03 |
| *** ggnel_t has quit IRC | 16:04 | |
| *** ykarel has quit IRC | 16:04 | |
| fungi | dhellmann: 69mb gzip-compressed | 16:04 |
| dhellmann | not to bad | 16:04 |
| fungi | dhellmann: i'll stick it somewhere for you to download | 16:04 |
| dhellmann | thanks, I'll close the question I opened | 16:04 |
| *** nadya has quit IRC | 16:05 | |
| dirk | clarkb: firewalld and susefirewall2 are the available options in opensuse right now. | 16:05 |
| *** nadya has joined #openstack-infra | 16:05 | |
| *** nadya has quit IRC | 16:05 | |
| *** prateek has joined #openstack-infra | 16:05 | |
| dirk | clarkb: I personally would love to have iptables-persistent in the distro going forward, but it's going to be a discussion and I don't have the bandwidth for that right now | 16:06 |
| clarkb | dirk: pabelanger I have +2'd buit didn't approve in case pabelanger wanted to review (he has been more involvedin the iptables conversion stuff hence my not approving) | 16:06 |
| clarkb | dirk: cool, I personally would like iptables-persistent as well :) | 16:06 |
| fungi | dhellmann: grab it from http://lists.openstack.org/pipermail/openstack/historical.txt.gz for now | 16:06 |
| dhellmann | downloading now | 16:07 |
| fungi | 268100 lines | 16:07 |
| openstackgerrit | Gábor Antal proposed openstack-dev/hacking master: Add a check for logger naming https://review.openstack.org/443250 | 16:07 |
| dhellmann | hmm, wc tells me 5881678 | 16:08 |
| dhellmann | md5sum gives 80fed0a32c3ad6be73b031939b423a9b for the .gz file | 16:08 |
| *** jpich has quit IRC | 16:08 | |
| fungi | dhellmann: oh, yep, i think view had only partially loaded it | 16:08 |
| dhellmann | ah | 16:08 |
| *** erikmwil_ has quit IRC | 16:08 | |
| fungi | 5881678 when i skip to the end | 16:08 |
| fungi | earliest message Jul 26 2010, last Jul 29 2013 | 16:09 |
| *** isaacb has quit IRC | 16:09 | |
| *** erikmwilson has joined #openstack-infra | 16:09 | |
| dhellmann | my client is importing the data | 16:09 |
| fungi | dhellmann: grep tells me it's 25537 messages total | 16:09 |
| dhellmann | wow | 16:10 |
| dhellmann | we're a chatty bunch | 16:10 |
| fungi | er, maybe only 25529 | 16:10 |
| fungi | my regex is probably not quite correct for matching message start lines | 16:10 |
| dhellmann | it's just ^From right? | 16:10 |
| fungi | well, '^From ' gets me 25537 but '^From .*@' only matches 25529 | 16:11 |
| dhellmann | oh, yeah, with the space | 16:11 |
| fungi | i can't remember if the separator is just lines starting with From[space] or [blank line]From[space] | 16:11 |
| fungi | and multi-line grep is annoying, so i didn't bother | 16:12 |
| *** jpich has joined #openstack-infra | 16:12 | |
| dhellmann | this will probably be easier to work with if we get it imported into mailman and let google index the pipermail pages :-) | 16:12 |
| fungi | yep | 16:12 |
| *** martinkopec has quit IRC | 16:13 | |
| *** Guest14948 is now known as ZZelle_ | 16:13 | |
| fungi | https://wiki.list.org/DOC/How%20do%20I%20import%20an%20archive%20into%20a%20new%20mailing%20list%3F has some tips | 16:13 |
| *** ociuhandu has joined #openstack-infra | 16:13 | |
| openstackgerrit | Jakub Libosvar proposed openstack-infra/project-config master: neutron: Add functional job to the gate queue https://review.openstack.org/460205 | 16:13 |
| *** gcb has quit IRC | 16:14 | |
| dhellmann | my original idea to get all of archives from all our lists into one place so I could more easily look at history. I'm not sure that's a realistic option, though. | 16:16 |
| clarkb | dhellmann: there are email ingesters for elasticsearhc and xapian and so on | 16:16 |
| dhellmann | clarkb : I'll look into those. sup uses xapian, so maybe I just throw everything into my sup database. | 16:17 |
| dhellmann | I wanted a browsing interface, more than a searching interface | 16:17 |
| *** sambetts is now known as sambetts|afk | 16:18 | |
| *** matrohon has quit IRC | 16:19 | |
| *** ijw has joined #openstack-infra | 16:21 | |
| *** john-davidge has quit IRC | 16:21 | |
| *** john-davidge has joined #openstack-infra | 16:21 | |
| *** Apoorva has joined #openstack-infra | 16:22 | |
| *** jaosorior_away is now known as jaosorior | 16:22 | |
| *** trown is now known as trown|lunch | 16:22 | |
| *** Hal has quit IRC | 16:24 | |
| *** tphummel has quit IRC | 16:25 | |
| rockstar | fungi, dhellmann, (it's been a while, but...) as I recall, I only moved the one list. No one could be sure what other lists needed to migrate. | 16:25 |
| *** jpena|brb is now known as jpena | 16:26 | |
| fungi | rockstar: no problem, thanks for the recollection! | 16:26 |
| fungi | rockstar: i found the general ml mbox dump from lp in reed's homedir, so working on cleaning that up for prepending now | 16:26 |
| *** jaosorior has quit IRC | 16:26 | |
| *** jaosorior has joined #openstack-infra | 16:27 | |
| *** jaosorior is now known as jaosorior_away | 16:27 | |
| fungi | dhellmann: the cleanarch found 25537 messages in that mbox, fwiw | 16:29 |
| fungi | so i guess my original grep was sufficient | 16:29 |
| openstackgerrit | Paul Belanger proposed openstack-infra/project-config master: Add jenkins-slave element for nodepool https://review.openstack.org/460137 | 16:29 |
| *** aarefiev is now known as aarefiev_afk | 16:30 | |
| *** Julien-zte has quit IRC | 16:30 | |
| *** martinkopec has joined #openstack-infra | 16:30 | |
| *** Julien-zte has joined #openstack-infra | 16:30 | |
| *** Julien-zte has quit IRC | 16:31 | |
| fungi | and cleanarch's output file matched the original, byte for byte, so i guess it was pre-cleaned or otherwise suitable | 16:31 |
| *** Julien-zte has joined #openstack-infra | 16:31 | |
| *** jamielennox is now known as jamielennox|away | 16:31 | |
| fungi | i need to pop out and grab some lunch, but will continue messing with this when i return | 16:31 |
| *** Julien-zte has quit IRC | 16:31 | |
| *** Julien-zte has joined #openstack-infra | 16:32 | |
| *** ijw has quit IRC | 16:32 | |
| *** Julien-zte has quit IRC | 16:32 | |
| *** martinkopec has quit IRC | 16:33 | |
| BobBall | mordred: Looking at http://paste.openstack.org/show/608085 I think you also need to set crtr_ver to 0x0 if it's not already set to 0? https://github.com/xapi-project/blktap/blob/master/vhd/lib/libvhd.c#L1263 which checks for the batmap (Citrix's VHD extension) will return no batmap if the version is < 1. | 16:33 |
| *** beslemon has quit IRC | 16:33 | |
| *** Julien-zte has joined #openstack-infra | 16:35 | |
| *** Julien-zte has quit IRC | 16:35 | |
| *** mhickey has joined #openstack-infra | 16:35 | |
| *** Julien-zte has joined #openstack-infra | 16:36 | |
| *** vhosakot has joined #openstack-infra | 16:37 | |
| *** jpich has quit IRC | 16:37 | |
| *** ijw has joined #openstack-infra | 16:37 | |
| *** florianf has quit IRC | 16:38 | |
| *** felipemonteiro_ has quit IRC | 16:39 | |
| *** ociuhandu has quit IRC | 16:40 | |
| *** lucasagomes is now known as lucas-afk | 16:40 | |
| *** Julien-zte has quit IRC | 16:40 | |
| *** adrian_otto has joined #openstack-infra | 16:40 | |
| *** ijw has quit IRC | 16:40 | |
| *** dizquierdo has quit IRC | 16:44 | |
| *** dprince has quit IRC | 16:46 | |
| *** dprince has joined #openstack-infra | 16:46 | |
| *** derekh has quit IRC | 16:46 | |
| openstackgerrit | Andreas Florath proposed openstack-infra/bindep master: WIP: Fix emerge testcases https://review.openstack.org/460217 | 16:47 |
| *** Sukhdev_ has quit IRC | 16:48 | |
| *** lamt has quit IRC | 16:48 | |
| clarkb | mordred: I saw https://github.com/Microsoft/azure-vhd-utils and got excited for a half second that maybe it would be useful. Nope | 16:48 |
| *** usertm has joined #openstack-infra | 16:48 | |
| clarkb | mordred: however its interesting that azure requires fixed vhd images | 16:49 |
| clarkb | mordred: which may help inform one of your earlier questions | 16:49 |
| *** BobBall is now known as BobBall_AWOL | 16:50 | |
| *** tesseract has quit IRC | 16:51 | |
| *** marst_ has quit IRC | 16:52 | |
| mordred | clarkb, BobBall_AWOL: http://paste.openstack.org/show/608106/ sets version (I think) | 16:54 |
| *** tphummel has joined #openstack-infra | 16:54 | |
| openstackgerrit | Andreas Florath proposed openstack-infra/bindep master: Fix emerge testcases https://review.openstack.org/460217 | 16:55 |
| *** ijw has joined #openstack-infra | 17:01 | |
| *** ociuhandu has joined #openstack-infra | 17:01 | |
| openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci master: CI test - never merge https://review.openstack.org/460223 | 17:04 |
| openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci master: CI test - never merge https://review.openstack.org/460223 | 17:04 |
| *** ijw has quit IRC | 17:05 | |
| *** dhinesh has joined #openstack-infra | 17:06 | |
| *** claudiub|2 has joined #openstack-infra | 17:08 | |
| *** ijw has joined #openstack-infra | 17:09 | |
| *** ociuhandu has quit IRC | 17:10 | |
| *** claudiub|3 has joined #openstack-infra | 17:11 | |
| *** claudiub has quit IRC | 17:11 | |
| *** john-davidge has quit IRC | 17:11 | |
| *** chem has joined #openstack-infra | 17:11 | |
| *** Qiming has quit IRC | 17:12 | |
| openstackgerrit | Merged openstack-infra/irc-meetings master: Add OpenStack-Helm community meeting https://review.openstack.org/459988 | 17:12 |
| *** esikachev has joined #openstack-infra | 17:13 | |
| *** claudiub|2 has quit IRC | 17:14 | |
| *** Qiming has joined #openstack-infra | 17:14 | |
| *** eharney has quit IRC | 17:15 | |
| *** chem has quit IRC | 17:15 | |
| *** ijw has quit IRC | 17:17 | |
| *** chem has joined #openstack-infra | 17:17 | |
| *** dtantsur is now known as dtantsur|afk | 17:17 | |
| openstackgerrit | Rosario Di Somma proposed openstack-infra/shade master: Add a _normalize_volume_backups method https://review.openstack.org/460127 | 17:17 |
| *** sshnaidm is now known as sshnaidm|afk | 17:19 | |
| *** Swami has quit IRC | 17:20 | |
| *** hamzy has joined #openstack-infra | 17:20 | |
| *** esberglu has quit IRC | 17:21 | |
| *** ociuhandu has joined #openstack-infra | 17:21 | |
| *** kornica has quit IRC | 17:23 | |
| dmsimard | In cores, would love an easy review on https://review.openstack.org/#/c/459913/ -- it should be the last iteration for the job to work properly, thanks | 17:25 |
| dmsimard | s/In/Hi/ | 17:25 |
| *** ociuhandu has quit IRC | 17:26 | |
| *** Rockyg has quit IRC | 17:26 | |
| openstackgerrit | Witold Bedyk proposed openstack-infra/project-config master: Set Monasca Java tempests to run with mod_wsgi https://review.openstack.org/460228 | 17:26 |
| asselin_ | clarkb, fyi regarding voluptous zuul issue...narrowed it down to an issue with my zuul.conf file...totally unexpected.... | 17:27 |
| asselin_ | i.e. it works with zuul.conf-sample | 17:28 |
| *** e0ne has quit IRC | 17:29 | |
| asselin_ | I guess this is why: https://git.openstack.org/cgit/openstack-infra/project-config/tree/tools/zuul.conf-sample#n33 I don't have those | 17:30 |
| *** yamahata_ has joined #openstack-infra | 17:30 | |
| *** eharney has joined #openstack-infra | 17:30 | |
| *** Apoorva_ has joined #openstack-infra | 17:30 | |
| pabelanger | clarkb: cmurphy: confirmed working for xenial: https://review.openstack.org/#/c/460137/ | 17:31 |
| clarkb | oh ya the connection data is cross checekd against your layout | 17:31 |
| clarkb | I ran into this on monday fixing zuul tests too | 17:31 |
| asselin_ | clarkb, ok yeah...b/c i'm upgrading from an old zuul...will convert this too. glad to get to the bottom of it! | 17:31 |
| *** makowals_ has joined #openstack-infra | 17:32 | |
| *** Apoorva has quit IRC | 17:33 | |
| *** makowals has quit IRC | 17:34 | |
| clarkb | pabelanger: one comment | 17:35 |
| *** esberglu has joined #openstack-infra | 17:35 | |
| *** esberglu has quit IRC | 17:36 | |
| *** esberglu_ has joined #openstack-infra | 17:36 | |
| clarkb | pabelanger: and can you review https://review.openstack.org/#/c/459055/3 since I didn't approve to make sure you had a chance to see it | 17:37 |
| pabelanger | +3 | 17:38 |
| *** trown|lunch is now known as trown | 17:40 | |
| *** pcaruana has quit IRC | 17:40 | |
| asselin_ | clarkb, I don't see any of those here though: https://git.openstack.org/cgit/openstack-infra/puppet-zuul/tree/templates/zuul.conf.erb is that expected? | 17:41 |
| *** beslemon has joined #openstack-infra | 17:42 | |
| clarkb | asselin_: is it possibly a zuulv3 thing? | 17:43 |
| clarkb | our production zuul doesn't have them either and is happy | 17:43 |
| asselin_ | clarkb, yeah..maybe it hasn't been restated in a while? I don't think it's related to zuulv3 | 17:43 |
| *** jkilpatr_ has joined #openstack-infra | 17:44 | |
| openstackgerrit | Clint 'SpamapS' Byrum proposed openstack-infra/zuul feature/zuulv3: Ignore directories when copying job dir components https://review.openstack.org/460234 | 17:45 |
| *** jamesmca_ has quit IRC | 17:45 | |
| openstackgerrit | David Shrewsbury proposed openstack-infra/zuul feature/zuulv3: Add a finger protocol log streamer https://review.openstack.org/456721 | 17:46 |
| openstackgerrit | David Shrewsbury proposed openstack-infra/zuul feature/zuulv3: Allow for specifying root job directory https://review.openstack.org/456691 | 17:46 |
| *** jkilpatr has quit IRC | 17:46 | |
| openstackgerrit | Merged openstack-infra/project-config master: Add firewall support for opensuse https://review.openstack.org/459055 | 17:46 |
| pabelanger | clarkb: replied to 460137 | 17:47 |
| *** jpena is now known as jpena|off | 17:47 | |
| *** csomerville has quit IRC | 17:49 | |
| *** adrian_otto has quit IRC | 17:49 | |
| *** csomerville has joined #openstack-infra | 17:49 | |
| *** gaurangt has joined #openstack-infra | 17:50 | |
| *** shardy has quit IRC | 17:50 | |
| *** tosky has quit IRC | 17:51 | |
| *** ociuhandu has joined #openstack-infra | 17:51 | |
| gaurangt | hi, can anyone help approve the patch https://review.openstack.org/#/c/459241/ for driverlog changes?\ | 17:52 |
| clarkb | gaurangt: https://review.openstack.org/#/admin/groups/320,members are the core reviewers for that project | 17:53 |
| pabelanger | asselin_: clarkb: see the patch from jeblair for connection objects in puppet-zuul: https://review.openstack.org/#/c/459728/ | 17:54 |
| fungi | dhellmann: further reading points out that prepending the imported messages to the current list's mbox file will change message numbers and invalidate old urls. old messages can apparently be appended instead as long as we don't care that the message numbers will be non-chronological where we've inserted the old set (which seems like a not particularly terrible side effect) | 17:54 |
| clarkb | pabelanger: that doesn't explain why it fails though | 17:54 |
| clarkb | pabelanger: they are deprected but should still work except for in zuulv3 aiui | 17:54 |
| dhellmann | fungi : will the index show them in chronological order, even though the numbers are not assigned that way? | 17:55 |
| *** hongbin has joined #openstack-infra | 17:55 | |
| fungi | dhellmann: yes, it looks like when we tell mailman to rebuild the pipermail indices those will all be correctly ordered/sorted, the numbering will just be non-chronological between the original and imported messages | 17:55 |
| dhellmann | fungi : ok, that seems like it should be fine, then | 17:55 |
| *** ociuhandu has quit IRC | 17:55 | |
| *** efoley_ has quit IRC | 17:56 | |
| dhellmann | I want to be able to click next next next but I don't care what names/numbers the pages have | 17:56 |
| gaurangt | clarkb, thanks | 17:57 |
| fungi | dhellmann: though looking at how pipermail organizes message numbers bucketed by month, maybe the impact from the prepend method isn't especially onerous either | 17:57 |
| asselin_ | clarkb, pabelanger deleting those lines does cause zuul-server -t to fail: http://paste.openstack.org/show/608113/ (confirming in case there was something else) | 17:57 |
| fungi | jeblair: any opinion on whether we should prepend or append the old lp ml dump to the mailman mbox file? | 17:57 |
| dhellmann | fungi : my impression is that we have a lot of places where we share links to old messages, so I would hate to invalidate all of those pages | 17:57 |
| fungi | dhellmann: yeah, but the numbering seems to start over each month. e.g. http://lists.openstack.org/pipermail/openstack-announce/2015-December/000869.html | 17:58 |
| dhellmann | oh, that's a surprise | 17:58 |
| *** lamt has joined #openstack-infra | 17:58 | |
| dhellmann | it would minimize the impact, then, as you say | 17:58 |
| fungi | so it's indexed by number of messages within that month | 17:58 |
| fungi | at worst we have a few messages overlap in the month where we moved from the lp ml to lists.o.o | 17:58 |
| fungi | so they would be reordered | 17:58 |
| * dhellmann nods | 17:59 | |
| jeblair | fungi: your latest information is drawing me toward prepend. it might be worth a dry run to double check? | 17:59 |
| jeblair | (and sorry, i don't have more to add from experience) | 17:59 |
| fungi | jeblair: not sure how to do a dry run with the mm arch --wipe command and still be able to evaluate the files | 18:00 |
| fungi | though i could easily tar up a copy right after stopping mm and then we can evaluate the result and undo it if we want before starting back up | 18:00 |
| openstackgerrit | Morgan Fainberg proposed openstack-infra/shade master: Convert test_role_assignments to requests mock https://review.openstack.org/448313 | 18:00 |
| *** electrofelix has quit IRC | 18:01 | |
| jeblair | fungi: was thinking maybe on another host with a copy of the data? but yeah, the undo approach seems workable. :) we can certainly offline mm for 30-60 mins without significant impact. | 18:01 |
| openstackgerrit | Paul Belanger proposed openstack-infra/system-config master: Remove puppet-jenkins from single_use_slave.pp https://review.openstack.org/460247 | 18:03 |
| fungi | dhellmann: jeblair: the earliest message in the current mbox is Jul 25 2013 and the last in the lp dump is Jul 29 2013 so there will be a little non-chronology with the prepend method as well (about 4 days worth) | 18:03 |
| dhellmann | I wonder if any of those are duplicates? I only did a spot check. | 18:04 |
| fungi | i've found some mbox merge scripts which could interleave the messages from the overlapping range | 18:04 |
| fungi | if we really, _really_ care about that | 18:04 |
| dhellmann | I don't think we do | 18:04 |
| * fungi personally doesn't | 18:04 | |
| *** nadya has joined #openstack-infra | 18:05 | |
| dhellmann | these messages are old enough that I'm prepared to forego the most pedantic solutions :-) | 18:05 |
| dhellmann | having them online at all will be good enough | 18:05 |
| jeblair | fungi: i don't. i think the url permanence is the main thing to worry about, but i also think we can live with that 1-month range of urls changing. | 18:05 |
| dhellmann | jeblair : ++ | 18:05 |
| fungi | and really it's only ~6 days of urls changing | 18:05 |
| *** cshastri has quit IRC | 18:05 | |
| fungi | because the list started on the new server very late in the month | 18:06 |
| fungi | (at least if my theory is correct) | 18:06 |
| jeblair | fungi: won't that mean nearly a month of new message numbers show up before the current #1? | 18:06 |
| fungi | okay, i'll write up a brief maintenance plan in an etherpad we can refer to including checkpointing and rollback | 18:06 |
| *** srwilkers has quit IRC | 18:06 | |
| *** nadya has quit IRC | 18:07 | |
| fungi | jeblair: right, but that's just the messages from july 25 through 31 whose urls will change | 18:07 |
| jeblair | fungi: gotcha | 18:08 |
| fungi | so nearly a month of new messages, yes, but the messages impacted by changing urls only span ~6 days | 18:08 |
| *** srwilkers has joined #openstack-infra | 18:09 | |
| openstackgerrit | David Shrewsbury proposed openstack-infra/zuul feature/zuulv3: Add a finger protocol log streamer https://review.openstack.org/456721 | 18:10 |
| openstackgerrit | Paul Belanger proposed openstack-infra/project-config master: [WIP] Remove puppet from nodepool images https://review.openstack.org/460252 | 18:10 |
| *** john-davidge has joined #openstack-infra | 18:13 | |
| pabelanger | AJaeger: mind reviewing https://review.openstack.org/#/c/460137/ | 18:13 |
| *** rossella_s has quit IRC | 18:13 | |
| *** baoli has quit IRC | 18:13 | |
| *** knangia has joined #openstack-infra | 18:14 | |
| *** john-davidge has quit IRC | 18:17 | |
| *** ralonsoh has quit IRC | 18:17 | |
| *** tosky has joined #openstack-infra | 18:18 | |
| *** beslemon has quit IRC | 18:18 | |
| *** EricGonczer_ has joined #openstack-infra | 18:19 | |
| clarkb | fungi: I'm putting together summit schedule for myself and have the infra onboarding and stackalytics sessions on there. Any other sessions that are particularly infray that you know of? | 18:20 |
| *** jamesmca_ has joined #openstack-infra | 18:20 | |
| fungi | clarkb: firehose talk is thursday at 4:10pm | 18:20 |
| *** baoli has joined #openstack-infra | 18:21 | |
| pabelanger | I might miss firehose :( need to get to airport early | 18:21 |
| pabelanger | will be there for both stackalytics and onboarding | 18:21 |
| fungi | clarkb: also a storyboard 101 survival guide for the migrated on wednesday at 1:50pm | 18:21 |
| pabelanger | fungi: did you say there was a schedule up for possible dinner night? | 18:22 |
| dhellmann | clarkb : your background with logstash may mean you have useful input into https://www.openstack.org/summit/boston-2017/summit-schedule/events/18507/logging-working-group-working-session | 18:22 |
| dhellmann | clarkb : though I don't know if you have the interest, and that's OK, too :-) | 18:22 |
| fungi | pabelanger: i did not, but we should consider it for sure | 18:22 |
| pabelanger | k, I can start a ML post if needed | 18:22 |
| pabelanger | not sure about locations to eat | 18:22 |
| openstackgerrit | Sagi Shnaidman proposed openstack-infra/tripleo-ci master: DONT REVIEW: test image building https://review.openstack.org/459786 | 18:24 |
| *** ijw has joined #openstack-infra | 18:24 | |
| *** nadya has joined #openstack-infra | 18:25 | |
| *** andreas-f has quit IRC | 18:25 | |
| *** gyee has joined #openstack-infra | 18:26 | |
| clarkb | dhellmann: ya I think I grabbed the logging ones I saw, let me check that one is there | 18:28 |
| dhellmann | clarkb : cool | 18:28 |
| *** sirushti has quit IRC | 18:28 | |
| *** e0ne has joined #openstack-infra | 18:28 | |
| *** ijw has quit IRC | 18:29 | |
| *** adrian_otto has joined #openstack-infra | 18:30 | |
| *** jamesmca_ has quit IRC | 18:33 | |
| clarkb | I've added firehose and storyboard sessions, thanks | 18:34 |
| clarkb | I've got a talk on surviving python packaging that I should go write | 18:35 |
| clarkb | it might be fun to just have ot be a sequence of package install commands that all fail until the end | 18:35 |
| *** Apoorva_ has quit IRC | 18:37 | |
| clarkb | fungi: speaking of firehose https://review.openstack.org/#/c/460194/ could use a second review | 18:37 |
| *** jkilpatr_ has quit IRC | 18:37 | |
| *** Apoorva has joined #openstack-infra | 18:38 | |
| *** jkilpatr has joined #openstack-infra | 18:39 | |
| *** MasterOfBugs has quit IRC | 18:40 | |
| *** baoli has quit IRC | 18:42 | |
| *** nadya has quit IRC | 18:43 | |
| *** e0ne has quit IRC | 18:44 | |
| *** baoli has joined #openstack-infra | 18:48 | |
| pabelanger | for some reason, one of the nodes in infracloud-chocolate is launching a DIB from 2016-12-14 17:03 | 18:48 |
| pabelanger | going to see if I can figure out why | 18:48 |
| *** marst_ has joined #openstack-infra | 18:49 | |
| openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci master: Remove centos-release-ceph-* package provided by nodepool https://review.openstack.org/460277 | 18:52 |
| *** usertm has quit IRC | 18:52 | |
| fungi | dhellmann: jeblair: i was wrong... message numbers continue to increment across month boundaries after all, so we should probably only append the lp dump instead | 18:52 |
| * fungi should have checked that more closely earlier | 18:52 | |
| dhellmann | fungi : oh, do they count up to some value before resetting? or are they counted within a given archive file, regardless of the month? | 18:53 |
| fungi | dhellmann: the latter. e.g. http://lists.openstack.org/pipermail/openstack/2017-April/019275.html is the 19275 message in the current archive's mbox | 18:53 |
| dhellmann | got it | 18:54 |
| fungi | so i'm adjusting the plan i'm writing to use the append option instead | 18:54 |
| dhellmann | and I guess the archive file rolls over at a given size? | 18:54 |
| fungi | that's a great question. i don't know what happens when it hits a million messages | 18:55 |
| dhellmann | do we have more than one mbox file now? | 18:55 |
| fungi | doesn't appear so, no | 18:55 |
| dhellmann | ah | 18:55 |
| *** ociuhandu has joined #openstack-infra | 18:55 | |
| dhellmann | well, then, yeah, I guess we need to append | 18:56 |
| AJaeger | pabelanger: +3 | 18:56 |
| *** ssbarnea has quit IRC | 18:57 | |
| openstackgerrit | Merged openstack-infra/shade master: Add a _normalize_volume_backups method https://review.openstack.org/460127 | 18:58 |
| *** EricGonczer_ has quit IRC | 18:59 | |
| *** Hal has joined #openstack-infra | 18:59 | |
| clarkb | pabelanger: if openstack invented time travel please let us know | 19:00 |
| pabelanger | clarkb: ya, this is weird | 19:00 |
| pabelanger | | 0a74a968-d24e-4afb-add0-14f234c7fe18 | ubuntu-xenial-infracloud-chocolate-8607644 | DELETED | None | NOSTATE | | ubuntu-xenial-1493185431 | 8bf7ceb9-2d7b-46a6-b6a0-0a2b55845819 | nova | compute015.chocolate.ic.openstack.org | | | 19:00 |
| pabelanger | was the image | 19:00 |
| *** dtardivel has quit IRC | 19:00 | |
| pabelanger | err, server | 19:00 |
| pabelanger | 8bf7ceb9-2d7b-46a6-b6a0-0a2b55845819 is the latest ubuntu-xenial image too | 19:00 |
| pabelanger | so, I have no idea why kvm would use something from 2016 | 19:00 |
| clarkb | bug in nova's image caching maybe? | 19:01 |
| openstackgerrit | Emilien Macchi proposed openstack-infra/project-config master: Remove centos openstack and ceph repos https://review.openstack.org/459425 | 19:01 |
| pabelanger | clarkb: maybe, will have to dig more | 19:01 |
| *** MasterOfBugs has joined #openstack-infra | 19:02 | |
| EmilienM | AJaeger, pabelanger: https://review.openstack.org/459425 this is an outstanding request, please :-) | 19:02 |
| *** EricGonczer_ has joined #openstack-infra | 19:02 | |
| mtreinish | clarkb: I'm curious which version of openssl is installed on the subunit workers? | 19:03 |
| AJaeger | EmilienM: project-config needs some review love.... | 19:03 |
| mtreinish | clarkb: I wonder if trusty doesn't even have 1.0.1 (I find that unlikely though) | 19:03 |
| openstackgerrit | Merged openstack-infra/project-config master: Add jenkins-slave element for nodepool https://review.openstack.org/460137 | 19:03 |
| *** jamesmca_ has joined #openstack-infra | 19:03 | |
| EmilienM | AJaeger: yeah :( | 19:03 |
| clarkb | mtreinish: Version: 1.0.1-4ubuntu5.39 is 01 | 19:05 |
| mtreinish | hmm that should also be new enough | 19:05 |
| clarkb | mtreinish: Version: 1.0.1f-1ubuntu2.22 is o2 | 19:05 |
| mtreinish | for tlsv1 | 19:05 |
| clarkb | mtreinish: ya I think we just need the change to merge then try it? | 19:06 |
| mtreinish | clarkb: I was just reading the paho code and it's sets tlsv1 by default in a bunch of places | 19:06 |
| mtreinish | so I'm not sure it'll make a difference | 19:06 |
| mtreinish | clarkb: https://github.com/eclipse/paho.mqtt.python/blob/v1.2.3/src/paho/mqtt/publish.py#L159-L180 | 19:06 |
| clarkb | mtreinish: and is that in whatever version we are running? | 19:07 |
| mtreinish | I have it install the latest from pip which should be 1.2.3 | 19:08 |
| mtreinish | https://github.com/openstack-infra/puppet-subunit2sql/blob/master/manifests/init.pp#L36 | 19:08 |
| *** ramishra has quit IRC | 19:08 | |
| mtreinish | there's no harm in trying it, I just don't expect it make a difference | 19:09 |
| fungi | jeblair: dhellmann: simple maintenance plan to merge the archives, including rollback: https://etherpad.openstack.org/p/lists.o.o-openstack-archive-import | 19:09 |
| *** jamesmca_ has quit IRC | 19:09 | |
| clarkb | mtreinish: ya its 1.2.3 confirmed | 19:10 |
| *** ramishra has joined #openstack-infra | 19:10 | |
| *** baoli has quit IRC | 19:10 | |
| *** baoli has joined #openstack-infra | 19:12 | |
| clarkb | mtreinish: you're secretly trying to get me to set a pdb breakpoint and check aren't you :P | 19:14 |
| mtreinish | clarkb: well reading the code I figured out why it's looping: https://github.com/eclipse/paho.mqtt.python/blob/v1.2.3/src/paho/mqtt/publish.py#L65-L70 | 19:15 |
| mtreinish | clarkb: heh we can do that too :) | 19:15 |
| mtreinish | I was gonna suggest a tcpdump to see what it's actually sending | 19:15 |
| clarkb | tcpdump is hard because its ssl | 19:16 |
| mtreinish | we should be able to tell the protocol at least right? | 19:16 |
| mtreinish | like sslv3, tls, or nothing | 19:16 |
| clarkb | ya I think we can get some basic info, give me a sec | 19:16 |
| pabelanger | ianw: re: http://git.openstack.org/cgit/openstack-infra/system-config/tree/install_puppet.sh#n160 I think we can remove that from DIB right? Or maybe do the same thing like EPEL, install it but disable it? | 19:17 |
| *** Kevin_Zheng has quit IRC | 19:17 | |
| clarkb | hrm the raw output actually isn't very readable, gonna hve to wireshark | 19:17 |
| *** esberglu_ has quit IRC | 19:21 | |
| *** marst_ has quit IRC | 19:22 | |
| clarkb | mtreinish: that doesn't help much, I mean I see MQTTnoisepahonoiseinfranoise | 19:22 |
| mtreinish | clarkb: is it encrypted at all? I would have expected the username (infra there) to be not readable | 19:24 |
| clarkb | mtreinish: wireshark identifies it as ssl data | 19:24 |
| mtreinish | ok | 19:24 |
| mtreinish | as long as the password is encrypted though, that's the whole reason we're bothering with ssl here | 19:25 |
| mtreinish | but that's not useful. I guess we're back to pdb breakpoint :) | 19:25 |
| fungi | wonder if it's doing opportunistic tls negotiation, a la starttls | 19:26 |
| strigazi | clarkb We still get "No route to host" from vms talking to keystone | 19:27 |
| clarkb | strigazi: can you provide an example? | 19:27 |
| openstackgerrit | Merged openstack-infra/project-config master: Remove centos openstack and ceph repos https://review.openstack.org/459425 | 19:27 |
| strigazi | clarkb http://logs.openstack.org/20/458720/8/check/gate-functional-dsvm-magnum-swarm-ubuntu-xenial/3ee67ac/logs/cluster-nodes/master-test_start_stop_container_from_api-172.24.5.6/cloud-init-output.txt.gz | 19:27 |
| strigazi | clarkb This is a vms trying to get a token from keystone | 19:28 |
| strigazi | clarkb This is a vm trying to get a token from keystone | 19:28 |
| *** jamesmca_ has joined #openstack-infra | 19:29 | |
| clarkb | strigazi: http://logs.openstack.org/20/458720/8/check/gate-functional-dsvm-magnum-swarm-ubuntu-xenial/3ee67ac/logs/iptables.txt.gz the problme is that iptables only allows port 5000 for keystone, not port 80 | 19:30 |
| clarkb | strigazi: so when devstack siwtched to uwsgi and no special ports it broke your job I think | 19:31 |
| mordred | ++ | 19:31 |
| *** ijw_ has joined #openstack-infra | 19:31 | |
| mordred | clarkb: thank you - I stared at logs for ages the other day (shade's magnum job started failing with that same error) but didn't think to look at iptables because I'm dumb | 19:31 |
| *** ijw_ has quit IRC | 19:32 | |
| *** ijw_ has joined #openstack-infra | 19:32 | |
| clarkb | mordred: you just need to wear a pager as person on call for datacenter networking, that'll make you look at firewalls real quick :) | 19:32 |
| jeblair | fungi: lgtm | 19:32 |
| strigazi | mordred, clarkb might know everything :) | 19:32 |
| clarkb | strigazi: also when you fix that you should allow 443 as well | 19:33 |
| clarkb | for the future when we enable tls everywhere or even just enable tls in your job and then enable only 443 | 19:33 |
| *** esikachev has quit IRC | 19:33 | |
| *** baoli has quit IRC | 19:34 | |
| *** ramishra has quit IRC | 19:36 | |
| strigazi | clarkb so I open only 80 and 443? | 19:37 |
| clarkb | strigazi: ya and stop opening 5000 since nothing listens there anymore | 19:37 |
| *** ramishra has joined #openstack-infra | 19:38 | |
| hongbin | hi, if anyone has a chance, need your help to kick off this job: https://review.openstack.org/#/c/458631/ | 19:39 |
| *** Qiming has quit IRC | 19:39 | |
| *** mdurrant has joined #openstack-infra | 19:40 | |
| *** ijw_ has quit IRC | 19:40 | |
| openstackgerrit | Jesse Keating proposed openstack-infra/zuul feature/zuulv3: Add support for github enterprise https://review.openstack.org/449258 | 19:40 |
| *** ijw has joined #openstack-infra | 19:40 | |
| *** Qiming has joined #openstack-infra | 19:42 | |
| *** mdurrant__ has quit IRC | 19:42 | |
| *** ramishra has quit IRC | 19:43 | |
| *** ijw has quit IRC | 19:44 | |
| fungi | https://jenkins.io/security/advisory/2017-04-26/ reminds me how happy i am that zuul v2.5 happened | 19:45 |
| *** eharney has quit IRC | 19:45 | |
| *** esikachev has joined #openstack-infra | 19:45 | |
| *** ramishra has joined #openstack-infra | 19:45 | |
| clarkb | "SECURITY-420: high" <- I laughed | 19:45 |
| fungi | indeed | 19:46 |
| * fungi wonders if that was intentional | 19:46 | |
| *** jamesmca_ has quit IRC | 19:48 | |
| *** shardy has joined #openstack-infra | 19:48 | |
| openstackgerrit | Merged openstack-infra/project-config master: Add a dsvm job for devstack-plugin-container https://review.openstack.org/458631 | 19:49 |
| *** esikachev has quit IRC | 19:49 | |
| *** ramishra has quit IRC | 19:51 | |
| mordred | clarkb: haha | 19:51 |
| clarkb | fungi: mtreinish notice that the cert for mqtt doesn't appear complete | 19:51 |
| clarkb | fungi: you added to hiera can you check that? | 19:51 |
| *** ramishra has joined #openstack-infra | 19:51 | |
| fungi | clarkb: yup, double-checking now | 19:52 |
| *** jamielennox|away is now known as jamielennox | 19:52 | |
| mtreinish | clarkb: it should be in public hiera now: https://review.openstack.org/#/c/375787/ | 19:52 |
| clarkb | fungi: its got no end certificate line on disk on the workers | 19:52 |
| clarkb | is the broken indentation for statusbot nicks breaking us maybe? | 19:53 |
| fungi | clarkb: hah! i think i got bitten by vi's copy buffer size | 19:53 |
| fungi | but as mtreinish points out. we can delete it from private hiera now anyway | 19:53 |
| clarkb | ok, my luinch just arried so I will leave that to toher people for now | 19:54 |
| pabelanger | clarkb: wow | 19:54 |
| clarkb | pabelanger: ? | 19:54 |
| pabelanger | clarkb | "SECURITY-420: high" <- I laughed | 19:54 |
| clarkb | ah | 19:55 |
| clarkb | :) | 19:55 |
| pabelanger | also, the advisory in general | 19:55 |
| mordred | yah | 19:55 |
| fungi | mtreinish: okay, it's removed from private hiera. assuming this corrects the pem, we can proceed to remove the same entry from our other private hiera groups/fqdns too | 19:55 |
| mtreinish | fungi: yep | 19:56 |
| mtreinish | granted the only other place it's set is on the logstash workers | 19:56 |
| *** ramishra has quit IRC | 19:56 | |
| *** baoli has joined #openstack-infra | 19:56 | |
| fungi | mtreinish: and on firehose01 | 19:57 |
| mtreinish | and they're not using it now (they will once we get a working pattern to copy from the subunit workers) | 19:57 |
| mtreinish | oh, yeah forgot about that | 19:57 |
| mtreinish | need to set the ca cert on the server :) | 19:58 |
| *** baoli has quit IRC | 19:59 | |
| *** baoli has joined #openstack-infra | 19:59 | |
| *** Apoorva has quit IRC | 19:59 | |
| *** eharney has joined #openstack-infra | 20:00 | |
| openstackgerrit | Ben Nemec proposed openstack-infra/tripleo-ci master: TEST: Bisect heat https://review.openstack.org/460189 | 20:00 |
| openstackgerrit | Ben Nemec proposed openstack-infra/tripleo-ci master: TEST: Bisect heat https://review.openstack.org/460188 | 20:00 |
| *** dprince has quit IRC | 20:00 | |
| *** esberglu has joined #openstack-infra | 20:00 | |
| *** adrian_otto has quit IRC | 20:02 | |
| *** pbourke has quit IRC | 20:02 | |
| *** ijw has joined #openstack-infra | 20:02 | |
| *** ramishra has joined #openstack-infra | 20:03 | |
| *** esikachev has joined #openstack-infra | 20:03 | |
| openstackgerrit | Matthew Treinish proposed openstack-infra/system-config master: Enable ansible callback events on firehose https://review.openstack.org/458248 | 20:03 |
| pabelanger | clarkb: just looking at the jenkins advisory more, I don't see backports for there 1.x releases | 20:03 |
| *** fandi has quit IRC | 20:04 | |
| pabelanger | clarkb: I guess that is EOL now? | 20:04 |
| fungi | nope! | 20:04 |
| *** cody-somerville has joined #openstack-infra | 20:04 | |
| *** cody-somerville has joined #openstack-infra | 20:04 | |
| *** esberglu has quit IRC | 20:05 | |
| pabelanger | fungi: firewalls up! | 20:05 |
| fungi | all except those "high" ports like 420/tcp, i guess? | 20:05 |
| pabelanger | :) | 20:06 |
| *** csomerville has quit IRC | 20:06 | |
| *** esikachev has quit IRC | 20:07 | |
| *** shardy has quit IRC | 20:07 | |
| *** ijw has quit IRC | 20:08 | |
| *** ramishra has quit IRC | 20:08 | |
| *** ijw has joined #openstack-infra | 20:09 | |
| *** ramishra has joined #openstack-infra | 20:12 | |
| *** matrohon has joined #openstack-infra | 20:15 | |
| *** rockyg has joined #openstack-infra | 20:16 | |
| *** andreww has joined #openstack-infra | 20:16 | |
| *** Apoorva has joined #openstack-infra | 20:17 | |
| *** xarses_ has quit IRC | 20:18 | |
| clarkb | mtreinish: fungi so once that goes out we need to restart mosquitto and the workers? | 20:19 |
| *** xarses_ has joined #openstack-infra | 20:19 | |
| mtreinish | clarkb: probably only the workers, mosquitto should already have the correct key | 20:19 |
| *** annegentle has quit IRC | 20:20 | |
| mtreinish | s/key/cert | 20:20 |
| *** andreww has quit IRC | 20:21 | |
| *** ramishra has quit IRC | 20:22 | |
| *** esikachev has joined #openstack-infra | 20:23 | |
| openstackgerrit | Jesse Keating proposed openstack-infra/zuul feature/zuulv3: Add trigger capability on github pr review https://review.openstack.org/449365 | 20:25 |
| *** EricGonczer_ has quit IRC | 20:27 | |
| *** esikachev has quit IRC | 20:28 | |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: [WIP] Fix nodepool integration job https://review.openstack.org/460299 | 20:29 |
| *** ramishra has joined #openstack-infra | 20:29 | |
| jeblair | easient review evar | 20:30 |
| fungi | clarkb: mtreinish: not sure but it seems to have updated now | 20:30 |
| clarkb | ya I restarted and still same connectivity issue | 20:31 |
| clarkb | its like we are fixing all the problems but hte actual problem | 20:31 |
| *** rockyg has quit IRC | 20:32 | |
| fungi | heh | 20:32 |
| clarkb | I have a mini test script from mtreinish to use to try and debug though | 20:33 |
| mordred | jeblair: I didn't realize you werw so good that you could fix problems with empty patches! | 20:34 |
| clarkb | mtreinish: your test script seems to work fine | 20:34 |
| *** megm has quit IRC | 20:34 | |
| clarkb | mtreinish: so I think its something specific to the subunit worker code | 20:34 |
| *** megm has joined #openstack-infra | 20:37 | |
| *** salv-orlando has joined #openstack-infra | 20:38 | |
| *** rockyg has joined #openstack-infra | 20:38 | |
| clarkb | mtreinish: looking at the code differences between the test script and work is the test script is normal "client" and the worker is pushmqtt | 20:39 |
| clarkb | is it possible the bug is there? or I guess just in the worker somewhere | 20:39 |
| *** ramishra has quit IRC | 20:39 | |
| *** Sukhdev has joined #openstack-infra | 20:39 | |
| jeblair | mordred: i figured i'd start with the minimal fix, then revise as needed! | 20:40 |
| *** ramishra has joined #openstack-infra | 20:41 | |
| clarkb | howd you commit an empty diff? | 20:41 |
| clarkb | this is trickery I need to learn | 20:41 |
| jeblair | clarkb: --allow-empty | 20:41 |
| clarkb | neat | 20:41 |
| Sukhdev | AJaeger : Hi Andreas, can you please look at this - https://review.openstack.org/#/c/457792/ | 20:42 |
| mtreinish | clarkb: the pushmqtt is a class wrapper around paho.mqtt.publish.publish_single which uses client internally | 20:44 |
| mtreinish | sry I was driving home | 20:44 |
| clarkb | ya I'm reading through it. publish.single is whats called | 20:45 |
| clarkb | is that the right name? | 20:45 |
| mtreinish | yeah | 20:45 |
| mtreinish | mine was a typo :) | 20:46 |
| *** dimtruck is now known as zz_dimtruck | 20:46 | |
| *** mat128 has quit IRC | 20:46 | |
| mtreinish | clarkb: https://github.com/eclipse/paho.mqtt.python/blob/master/src/paho/mqtt/publish.py#L186 | 20:46 |
| clarkb | mtreinish: the docstring there says tlsversion will default to none if not set? | 20:48 |
| *** ramishra has quit IRC | 20:48 | |
| clarkb | oh I see something one sec | 20:48 |
| mtreinish | clarkb: that's the dict. The lib is a bit weird that way | 20:48 |
| mtreinish | https://github.com/eclipse/paho.mqtt.python/blob/master/src/paho/mqtt/publish.py#L159-L180 is where it gets used | 20:49 |
| clarkb | it works! | 20:49 |
| *** zz_dimtruck is now known as dimtruck | 20:49 | |
| *** trown is now known as trown|outtypewww | 20:49 | |
| clarkb | mtreinish: its mqtt-ca-certs not mqtt-ca_certs in the yaml file | 20:49 |
| mtreinish | :( well that's on me | 20:49 |
| mtreinish | \o/ its working nwo though | 20:50 |
| fungi | oh, in subunit-woker.yaml? | 20:50 |
| clarkb | I've restarted both workers now with that fix in place | 20:50 |
| fungi | i see that now | 20:50 |
| clarkb | let me push a patch | 20:50 |
| mtreinish | http://paste.openstack.org/show/608138/ | 20:50 |
| mtreinish | I think I'll need to change the payload to include the filename too, because we're pushing for both with gz and without it looks like | 20:51 |
| fungi | doesn't look fixed in /etc/subunit2sql/subunit-woker.yaml on subunit-worker02 yet | 20:51 |
| *** esikachev has joined #openstack-infra | 20:51 | |
| clarkb | fungi: because that file isn't used | 20:51 |
| fungi | hah | 20:51 |
| clarkb | fungi: the jenkins-*worker.yaml is used | 20:51 |
| clarkb | notice the typo in the filename too | 20:51 |
| clarkb | what repo is this in? | 20:52 |
| fungi | yup | 20:52 |
| * mtreinish can't ever type correctly | 20:52 | |
| clarkb | mtreinish: I have the same problem | 20:52 |
| mtreinish | clarkb: it's either in system-config or puppet-subunit2sql | 20:52 |
| mtreinish | not sure which file you're looking at | 20:52 |
| mtreinish | /etc/subunit2sql/subunit-worker.yaml will be in systemconfig | 20:52 |
| fungi | looks like we're still using /etc/subunit2sql/jenkins-subunit-worker-A.yaml though | 20:53 |
| mtreinish | and the one with the suffix is from puppet-subunit2sql (the file in system-config file gets passed in as the source for the one with a suffix via a variable) | 20:53 |
| fungi | aha, got it | 20:54 |
| openstackgerrit | Clark Boylan proposed openstack-infra/system-config master: Fix use of _ that should be - in mqtt-ca_certs https://review.openstack.org/460307 | 20:54 |
| clarkb | mtreinish: ^ how is that? | 20:54 |
| mtreinish | clarkb: LGTM | 20:55 |
| *** ramishra has joined #openstack-infra | 20:55 | |
| *** esikachev has quit IRC | 20:55 | |
| ianw | pabelanger: re rdo removal ... iirc it was put there as a convenience for openvswitch, which was doing setup before tests etc. i don't know if it makes sense to disable by default. the reason i like making us think about epel stuff is because its *not* in rdo | 20:56 |
| clarkb | ianw: pabelanger yes rdo is where we get ovs to do multinode overlays and such | 20:56 |
| pabelanger | right, but we need to remove that from install_puppet.sh | 20:56 |
| pabelanger | since we'll be deleting that shortly | 20:57 |
| clarkb | why? | 20:57 |
| fungi | if devstack-gate uses it, devstack-gate can add it i guess | 20:57 |
| pabelanger | puppet element removal | 20:57 |
| clarkb | I see, its less about removing that specific thing and more about removing the script that pulls it in | 20:57 |
| pabelanger | so, does it make sense to add it to DIB element, or devstack-gate? | 20:57 |
| mordred | clarkb: there was a time when I compiled C code all day long for work - doing it now makes me wonder if computers have gotten slower in the last 8 years | 20:57 |
| clarkb | installing packages in d-g is such :( | 20:57 |
| ianw | i think i'd probably just prefer to have it as base on centos nodes | 20:57 |
| fungi | so dib then | 20:57 |
| clarkb | mordred: hahahaha I was actually thinking they got faster when compiling python. I remember when it took hours to compile language related things | 20:58 |
| pabelanger | ianw: issue is, that broke puppet / tripleo | 20:58 |
| pabelanger | ianw: first thing they do it remove it | 20:58 |
| clarkb | mordred: I maintained local ghc packages for solaris at one time that was so slow | 20:58 |
| fungi | mordred: or optimizations have gotten more aggressive keeping pace with advances in hardware | 20:58 |
| pabelanger | ianw: so, I do like the idea, where we install it, then disable the repo, and if the job needs it, they can enable-repo again | 20:58 |
| pabelanger | like we did for epel | 20:58 |
| pabelanger | it seems to work pretty well | 20:58 |
| clarkb | pabelanger: where are puppet and tripleo getting their packages if not rdo? I thought tahts why we have the mirror rdo problem | 20:59 |
| mordred | I also may have been spoiled by working on a project that spent a good amount of time optimizing its build ... | 20:59 |
| pabelanger | clarkb: depends on the branch, but for the most part their jobs setup the repo dependencys as part of the install process | 20:59 |
| pabelanger | clarkb: so, they like having nothing to ensure repo setup works | 21:00 |
| clarkb | mordred: oh ya that part python seemed to do poorly and rebuilt everything everytime I built it | 21:00 |
| clarkb | mordred: I was slightly confused | 21:00 |
| clarkb | pabelanger: so we're removing it so that they can add it back again? | 21:00 |
| pabelanger | clarkb: basically | 21:00 |
| clarkb | (I just want to udnerstand what the actual problem is) | 21:00 |
| pabelanger | clarkb: they also pin to specific repos too, again depending on the branch | 21:00 |
| *** Goneri has quit IRC | 21:01 | |
| *** knangia has quit IRC | 21:01 | |
| ianw | hmm, they want the [rdo|centos]-release* package *not* installed? | 21:02 |
| pabelanger | so, I think we could create something like: http://git.openstack.org/cgit/openstack/diskimage-builder/tree/diskimage_builder/elements/epel/pre-install.d/05-rpm-epel-release for RDO / centos repos. So far it is working well for epel | 21:02 |
| pabelanger | ianw: yes | 21:02 |
| *** rbrndt has quit IRC | 21:02 | |
| clarkb | mtreinish: looks like something restarted it, guessing puppet when it rewrote teh config file back to the typo'd version so the workers stopped again. But once my fix is in they should be happy | 21:02 |
| pabelanger | ianw: https://review.openstack.org/#/c/459425/ | 21:02 |
| ianw | well i'm not too sure about that, since re-installing that package will overwrite and destroy any mirror setup? | 21:02 |
| pabelanger | ianw: they have logic to account for that | 21:02 |
| ianw | that seems to be a gun pointed at everyone's foot | 21:02 |
| pabelanger | also, we don't populate mirror info for ceph and other repo files today | 21:03 |
| pabelanger | just CentOS-Base | 21:03 |
| pabelanger | so, if we did the element, we could fix that too | 21:03 |
| openstackgerrit | Matthew Treinish proposed openstack-infra/puppet-subunit2sql master: Add source_url to mqtt payload https://review.openstack.org/460313 | 21:03 |
| mtreinish | clarkb: ok cool | 21:03 |
| *** rbrndt has joined #openstack-infra | 21:04 | |
| *** annegentle has joined #openstack-infra | 21:05 | |
| *** ihrachys_ has joined #openstack-infra | 21:06 | |
| *** srwilkers has quit IRC | 21:07 | |
| ianw | pabelanger: as a base pov, if we're setting up mirrors (which we are) then i think we should pre-install the packages that provide the .repo files so jobs don't pull them in and overwrite the mirror config | 21:08 |
| *** ijw has quit IRC | 21:08 | |
| *** ihrachys has quit IRC | 21:09 | |
| pabelanger | ianw: sure, up until now, most jobs have done it them self. I see no reason doing that moving forward. I maybe don't think we should enable extra repos by default, let jobs opt into them | 21:09 |
| *** kgiusti has left #openstack-infra | 21:10 | |
| pabelanger | I can start on a DIB element tomorrow and get a review up | 21:10 |
| *** rhallisey has quit IRC | 21:10 | |
| *** ijw has joined #openstack-infra | 21:10 | |
| clarkb | fwiw I still don't understand why ovs is part of rdo and not base os. its not openstack specific | 21:10 |
| pabelanger | I seem to remember hearing it is on the road map for base | 21:11 |
| pabelanger | clarkb: ianw: cmurphy: also, I think we'll be ready to give https://review.openstack.org/#/c/460247/ a go tomorrow. Basically the last puppet bits we run inside DIB | 21:11 |
| ianw | pabelanger: it's just that basically nothing works without at least the qemu-ev packages | 21:12 |
| ianw | i guess, the argument is that devstack can get that right | 21:12 |
| *** annegentle has quit IRC | 21:12 | |
| openstackgerrit | Merged openstack-infra/system-config master: Fix use of _ that should be - in mqtt-ca_certs https://review.openstack.org/460307 | 21:12 |
| *** shrircha has joined #openstack-infra | 21:13 | |
| pabelanger | ianw: ya, I think we can ensure devstack works properly, and puppet / tripleo already manages it them self. | 21:13 |
| *** ihrachys_ has quit IRC | 21:14 | |
| pabelanger | I was planning on building an image and running reproduce.sh on it for centos to test this all out | 21:14 |
| pabelanger | maybe friday | 21:14 |
| *** askb has joined #openstack-infra | 21:14 | |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Catch and log url pattern formatting errors https://review.openstack.org/450468 | 21:14 |
| clarkb | pabelanger: keep in mind that ovs is only with a subset of jobs | 21:14 |
| clarkb | or rather the external to openstack ovs is | 21:14 |
| pabelanger | k | 21:14 |
| clarkb | and when you take that to run on centos I think its even smaller, puppet is one of them iirc and maybe kolla | 21:15 |
| *** ldnunes has quit IRC | 21:15 | |
| cmurphy | pabelanger: \o/ | 21:15 |
| *** ramishra has quit IRC | 21:15 | |
| *** thorst_afk has quit IRC | 21:16 | |
| pabelanger | clarkb: k, I'll reach out to both projects | 21:16 |
| clarkb | I don't think devstack has multinode jobs on centos | 21:17 |
| *** ramishra has joined #openstack-infra | 21:17 | |
| *** blancos has joined #openstack-infra | 21:17 | |
| *** shrircha has quit IRC | 21:17 | |
| openstackgerrit | Matthew Treinish proposed openstack-infra/puppet-log_processor master: Add MQTT support to the gearman worker https://review.openstack.org/449294 | 21:18 |
| *** jamesmca_ has joined #openstack-infra | 21:19 | |
| *** thorst_afk has joined #openstack-infra | 21:20 | |
| ianw | no, but wouldn't like to make it hard(er) | 21:20 |
| *** matrohon has quit IRC | 21:21 | |
| *** ijw has quit IRC | 21:21 | |
| *** ramishra has quit IRC | 21:21 | |
| *** ramishra has joined #openstack-infra | 21:22 | |
| *** zhenguo has quit IRC | 21:22 | |
| pabelanger | http://paste.openstack.org/show/608143/ | 21:23 |
| pabelanger | puppet noop! | 21:23 |
| *** thorst_afk has quit IRC | 21:24 | |
| cmurphy | omg | 21:25 |
| *** yolanda has joined #openstack-infra | 21:25 | |
| *** jamesmca_ has quit IRC | 21:26 | |
| *** esikachev has joined #openstack-infra | 21:26 | |
| openstackgerrit | Jesse Keating proposed openstack-infra/zuul feature/zuulv3: Add support for requiring github pr head status https://review.openstack.org/449390 | 21:27 |
| *** esberglu has joined #openstack-infra | 21:27 | |
| Sukhdev | AJaeger : are you around? | 21:28 |
| clarkb | Sukhdev: I doubt it he is in europe so it is very late at night now | 21:28 |
| clarkb | cmurphy is fueled by monster and doesnt' count | 21:29 |
| cmurphy | i'm literally getting in bed now | 21:29 |
| fungi | good call | 21:29 |
| Sukhdev | clarkb : oh I see - he mentioned he will approve this after a week - https://review.openstack.org/#/c/457792 | 21:30 |
| *** Hal has quit IRC | 21:31 | |
| *** esikachev has quit IRC | 21:31 | |
| fungi | Sukhdev: he also left you a comment that you reverted some fixes from an earlier patchset | 21:31 |
| fungi | so it needs correcting still | 21:31 |
| openstackgerrit | Sagi Shnaidman proposed openstack-infra/tripleo-ci master: DONT REVIEW: testing oooq https://review.openstack.org/459766 | 21:32 |
| *** eharney has quit IRC | 21:32 | |
| *** esikachev has joined #openstack-infra | 21:33 | |
| *** Apoorva_ has joined #openstack-infra | 21:34 | |
| Sukhdev | fungi : oh - darn it, I was looking at the older version of the path - thanks for pointing this out - let me fix the goof up | 21:35 |
| *** lamt has quit IRC | 21:35 | |
| *** ramishra has quit IRC | 21:35 | |
| *** Apoorva_ has quit IRC | 21:36 | |
| *** Apoorva_ has joined #openstack-infra | 21:36 | |
| *** Apoorva has quit IRC | 21:36 | |
| *** lamt has joined #openstack-infra | 21:37 | |
| *** ramishra has joined #openstack-infra | 21:38 | |
| *** esikachev has quit IRC | 21:39 | |
| openstackgerrit | Matthew Treinish proposed openstack-infra/system-config master: Add firehose schema doc for launchpad message https://review.openstack.org/454911 | 21:39 |
| openstackgerrit | Matthew Treinish proposed openstack-infra/system-config master: Add firehose schema docs https://review.openstack.org/454862 | 21:39 |
| openstackgerrit | Matthew Treinish proposed openstack-infra/system-config master: Add subunit gearman worker mqtt info to firehose docs https://review.openstack.org/460323 | 21:39 |
| *** jascott1 has quit IRC | 21:39 | |
| mtreinish | fungi: ^^^ should be some easy review | 21:39 |
| *** jascott1 has joined #openstack-infra | 21:40 | |
| openstackgerrit | James E. Blair proposed openstack-infra/nodepool feature/zuulv3: Fix zuul-nodepool integration test https://review.openstack.org/460325 | 21:40 |
| clarkb | mtreinish: subunit workers appear to be working now | 21:40 |
| clarkb | mtreinish: can you check on the application side? | 21:41 |
| mtreinish | clarkb: yeah it looks like it's all working and working through the backlog (its up to sat at 2am now) | 21:41 |
| mtreinish | clarkb: thanks for all the help | 21:41 |
| *** lamt has quit IRC | 21:42 | |
| *** dimtruck is now known as zz_dimtruck | 21:42 | |
| *** zz_dimtruck is now known as dimtruck | 21:42 | |
| *** jascott1_ has joined #openstack-infra | 21:42 | |
| *** ramishra has quit IRC | 21:43 | |
| mtreinish | clarkb: ready to move on to the next mqtt thing? :P | 21:43 |
| clarkb | uh | 21:43 |
| mtreinish | heh | 21:44 |
| *** jascott1 has quit IRC | 21:44 | |
| *** srobert has quit IRC | 21:44 | |
| clarkb | at some point I'm gonna have to close my virtual office door and prep for my summit talk | 21:44 |
| *** ramishra has joined #openstack-infra | 21:44 | |
| *** jcoufal_ has quit IRC | 21:44 | |
| mtreinish | clarkb: no worries (I need to do the same at some point too) | 21:45 |
| clarkb | mtreinish: at this point likely to be tomorrow though | 21:45 |
| clarkb | because running out of time today | 21:45 |
| *** jascott1_ has quit IRC | 21:46 | |
| mtreinish | clarkb: well for future reference the 2 big ones are: https://review.openstack.org/#/c/458248/ and https://review.openstack.org/#/c/449294/ | 21:46 |
| *** rockyg has quit IRC | 21:47 | |
| clarkb | does anyone want to pull the trigger on the stack at https://review.openstack.org/#/c/457823/ ? | 21:48 |
| *** chlong has quit IRC | 21:48 | |
| *** ijw has joined #openstack-infra | 21:48 | |
| clarkb | its not really important I guess. But would make things look more up to date | 21:48 |
| openstackgerrit | Jesse Keating proposed openstack-infra/zuul feature/zuulv3: Adds github triggering from status updates https://review.openstack.org/453844 | 21:49 |
| *** Goneri has joined #openstack-infra | 21:49 | |
| *** salv-orl_ has joined #openstack-infra | 21:49 | |
| mtreinish | clarkb: I'm going to miss the blue gradient in the upper left corner | 21:50 |
| clarkb | mtreinish: not sure if serious | 21:51 |
| *** jkilpatr has quit IRC | 21:51 | |
| *** ramishra has quit IRC | 21:52 | |
| openstackgerrit | Sukhdev Kapur proposed openstack-infra/project-config master: Adding new project networking-opencontrail https://review.openstack.org/457792 | 21:52 |
| *** jascott1 has joined #openstack-infra | 21:52 | |
| *** salv-orlando has quit IRC | 21:52 | |
| *** annegentle has joined #openstack-infra | 21:53 | |
| *** ramishra has joined #openstack-infra | 21:53 | |
| mtreinish | clarkb: only kind of | 21:56 |
| clarkb | hrm neighbor has a compressor going for a nail gun | 21:57 |
| clarkb | ELOUD | 21:57 |
| clarkb | mtreinish: https://review.openstack.org/#/c/449294/ is failing ci | 21:58 |
| mtreinish | oh, so it is | 21:59 |
| *** gouthamr has quit IRC | 21:59 | |
| mtreinish | hmm, http://logs.openstack.org/94/449294/5/check/gate-infra-puppet-apply-3-ubuntu-trusty/e6fee3f/puppetapplytest24.final.out.FAILED duplicate package install | 21:59 |
| openstackgerrit | Samantha Blanco proposed openstack-infra/project-config master: Add release note jobs to Patrole https://review.openstack.org/460333 | 22:00 |
| *** blancos has quit IRC | 22:00 | |
| mtreinish | ugh conflict between subunit2sql::worker and log_processor::init | 22:00 |
| mtreinish | I'll just if block it here I guess | 22:00 |
| openstackgerrit | Matthew Treinish proposed openstack-infra/puppet-log_processor master: Add MQTT support to the gearman worker https://review.openstack.org/449294 | 22:01 |
| *** Apoorva_ has quit IRC | 22:02 | |
| *** felipemonteiro__ has quit IRC | 22:03 | |
| *** Apoorva has joined #openstack-infra | 22:03 | |
| *** dhajare has quit IRC | 22:04 | |
| *** ramishra has quit IRC | 22:04 | |
| *** mdrabe has quit IRC | 22:04 | |
| openstackgerrit | Jeremy Stanley proposed openstack-infra/system-config master: Add firehose schema doc for launchpad message https://review.openstack.org/454911 | 22:05 |
| openstackgerrit | Jeremy Stanley proposed openstack-infra/system-config master: Add subunit gearman worker mqtt info to firehose docs https://review.openstack.org/460323 | 22:05 |
| openstackgerrit | Jeremy Stanley proposed openstack-infra/system-config master: Add firehose schema docs https://review.openstack.org/454862 | 22:05 |
| *** baoli has quit IRC | 22:05 | |
| fungi | mtreinish: ^ minor typo corrections | 22:05 |
| fungi | assuming you're good with those, happy to approve | 22:06 |
| *** ramishra has joined #openstack-infra | 22:06 | |
| *** esberglu has quit IRC | 22:06 | |
| mtreinish | fungi: LGTM | 22:07 |
| *** rfolco has quit IRC | 22:07 | |
| *** ijw has quit IRC | 22:10 | |
| *** ramishra has quit IRC | 22:12 | |
| openstackgerrit | Clint 'SpamapS' Byrum proposed openstack-infra/zuul feature/zuulv3: Add support for bwrap https://review.openstack.org/453851 | 22:12 |
| *** jamesdenton has quit IRC | 22:12 | |
| openstackgerrit | Clint 'SpamapS' Byrum proposed openstack-infra/zuul feature/zuulv3: Add support for bwrap https://review.openstack.org/453851 | 22:13 |
| *** ramishra has joined #openstack-infra | 22:13 | |
| *** salv-orl_ has quit IRC | 22:14 | |
| *** ijw has joined #openstack-infra | 22:14 | |
| *** john-davidge has joined #openstack-infra | 22:16 | |
| openstackgerrit | Clint 'SpamapS' Byrum proposed openstack-infra/zuul feature/zuulv3: Add support for bwrap https://review.openstack.org/453851 | 22:17 |
| openstackgerrit | Merged openstack-infra/system-config master: Add firehose schema docs https://review.openstack.org/454862 | 22:18 |
| *** jamesmca_ has joined #openstack-infra | 22:18 | |
| *** ijw has quit IRC | 22:18 | |
| *** claudiub|3 has quit IRC | 22:19 | |
| openstackgerrit | Merged openstack-infra/system-config master: Add firehose schema doc for launchpad message https://review.openstack.org/454911 | 22:20 |
| *** john-davidge has quit IRC | 22:20 | |
| *** ijw has joined #openstack-infra | 22:20 | |
| *** jamesmca_ has quit IRC | 22:23 | |
| *** jaypipes has quit IRC | 22:26 | |
| *** EricGonczer_ has joined #openstack-infra | 22:30 | |
| *** jkilpatr has joined #openstack-infra | 22:32 | |
| dhellmann | fungi, jeblair : that plan for importing the archives looks good to me, based on my reading of the MM wiki page | 22:34 |
| *** EricGonc_ has joined #openstack-infra | 22:34 | |
| *** EricGonczer_ has quit IRC | 22:34 | |
| *** hughhalf has quit IRC | 22:34 | |
| *** hughhalf has joined #openstack-infra | 22:35 | |
| abhi | clarkb, asselin, I could setup 3rd party CI correctly after I set java_args_override. Thanks | 22:36 |
| clarkb | abhi: cool, good to know that was the issue | 22:36 |
| *** vhosakot has quit IRC | 22:37 | |
| mtreinish | fungi: if you get a sec, this should be another simple mqtt review: https://review.openstack.org/#/c/458953/ | 22:39 |
| mtreinish | that should fix the metrics for statsd so it's proper counts instead of a running total | 22:40 |
| *** Goneri has quit IRC | 22:41 | |
| *** tonytan_brb has quit IRC | 22:42 | |
| *** esikachev has joined #openstack-infra | 22:45 | |
| *** EricGonc_ has quit IRC | 22:45 | |
| *** EricGonczer_ has joined #openstack-infra | 22:45 | |
| *** mhickey has quit IRC | 22:48 | |
| mtreinish | AJaeger: you asked about openstack health the other day, things should be catching up now. It's still working through the backlog, but it's almost done (its on the 24th right now) | 22:48 |
| *** esikachev has quit IRC | 22:49 | |
| openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci master: Remove centos-release-ceph-* package provided by nodepool https://review.openstack.org/460277 | 22:51 |
| *** esberglu has joined #openstack-infra | 22:52 | |
| openstackgerrit | Jesse Keating proposed openstack-infra/zuul feature/zuulv3: Implement pipeline requirement on github reviews https://review.openstack.org/453845 | 22:52 |
| *** rbrndt has quit IRC | 22:52 | |
| openstackgerrit | Clark Boylan proposed openstack-infra/elastic-recheck master: Track job timeouts in e-r https://review.openstack.org/460340 | 22:55 |
| clarkb | mtreinish: ^ thats the thing I promised from this morning | 22:55 |
| *** esberglu has quit IRC | 22:57 | |
| *** annegentle has quit IRC | 22:57 | |
| *** pradk has quit IRC | 22:59 | |
| mtreinish | clarkb: +A | 23:05 |
| *** gildub has joined #openstack-infra | 23:06 | |
| *** aeng has quit IRC | 23:07 | |
| *** adrian_otto has joined #openstack-infra | 23:07 | |
| *** ijw has quit IRC | 23:07 | |
| openstackgerrit | Merged openstack-infra/elastic-recheck master: Track job timeouts in e-r https://review.openstack.org/460340 | 23:08 |
| *** xarses_ has quit IRC | 23:08 | |
| *** ijw has joined #openstack-infra | 23:10 | |
| *** Apoorva_ has joined #openstack-infra | 23:10 | |
| *** Apoorva_ has quit IRC | 23:11 | |
| *** Apoorva has quit IRC | 23:11 | |
| *** Apoorva has joined #openstack-infra | 23:11 | |
| *** gordc has quit IRC | 23:12 | |
| clarkb | mtreinish: is http://logs.openstack.org/48/458548/1/gate/gate-grenade-dsvm-neutron-ubuntu-xenial/318dab4/logs/grenade.sh.txt.gz#_2017-04-26_22_27_03_937 a known issue? | 23:13 |
| openstackgerrit | Emilien Macchi proposed openstack-infra/project-config master: tripleo: EOLing Mitaka https://review.openstack.org/460347 | 23:13 |
| clarkb | I'm about to double check on our image in whatever region ^ ran in | 23:13 |
| *** dimtruck is now known as zz_dimtruck | 23:14 | |
| mtreinish | clarkb: that was a bug I caused a couple months ago trying to land a backport of the code to remove amazon images (the 3 part things) from devstack | 23:14 |
| mtreinish | I didn't think that landed | 23:14 |
| mtreinish | we shouldn't be hitting that in the regular grenade jobs | 23:15 |
| mtreinish | clarkb: the fix is: https://review.openstack.org/#/c/435106/ which will have to land in the backports too | 23:15 |
| clarkb | mtreinish: ok that just failed in the gate fwiw | 23:16 |
| mtreinish | oh, ianw has a -1 on that. I'll have to look at that and addess the comments tomorrow | 23:16 |
| clarkb | and cirros-0.3.4-x86_64-disk.img is in the files cache on the image there | 23:16 |
| clarkb | oh its failing on the name in glance nevermind | 23:17 |
| mtreinish | yeah the failure is caused by a mismatch between filename and image name in glance (devstack always strips the file extension) | 23:17 |
| mtreinish | but it's not an intermittent failure mode, it's an all or nothing deal | 23:17 |
| *** ramishra has quit IRC | 23:21 | |
| clarkb | mtreinish: note that the fail is in grenade which doesn't seem to be touched by your fix? | 23:22 |
| *** ramishra has joined #openstack-infra | 23:23 | |
| *** aeng has joined #openstack-infra | 23:24 | |
| *** Julien-zte has joined #openstack-infra | 23:24 | |
| mtreinish | clarkb: the bug is very weird, I can try to explain it, but it has to do with how weird the iamge upload and handling code in devstack is | 23:25 |
| clarkb | mtreinish: ok I trust you enough to nto worry about it :) | 23:25 |
| mtreinish | clarkb: bascially there are certain paths that devstack will always stip the extension from the name it uploads to glance (while others it doesnt) | 23:25 |
| clarkb | just curious | 23:26 |
| mtreinish | that doesn't get exported to the variables by default if it does the extension stripping | 23:26 |
| clarkb | mtreinish: gotcha so if name=foo.img, in some cases devstack uploads foo.img and gives it a name of foo in glance and doesn't change "name" | 23:26 |
| mtreinish | but for internal devstack operations (like configuring tempest) if it can't find the image from the variable DEFAULT_IMAGE_NAME it falls back to the list of all images and tries them in alphabetical order | 23:27 |
| clarkb | so then grenade goes boot using name and bam name doesn't exist | 23:27 |
| mtreinish | the problem was with grenade on the old side it uses that variable and it's not found | 23:27 |
| mtreinish | clarkb: right | 23:27 |
| mtreinish | we didn't catch it on master, because it always works since there is only ever cirros uploaded into glance | 23:27 |
| *** ramishra has quit IRC | 23:28 | |
| mtreinish | the octavia devstack plugin actually hit this bug too because they upload an additional image that starts with 'Amp' | 23:29 |
| mtreinish | so it was getting used instead of cirros and things started to fail | 23:29 |
| mtreinish | but anyway I'm not sure how that stable ocata patch is hitting this failure mode | 23:30 |
| johnsom | Did that ever get fixed? | 23:30 |
| mtreinish | johnsom: the patch for the fix fell through the cracks | 23:30 |
| mtreinish | I just pushed a new rev to fix ianw's comments | 23:30 |
| ianw | do i need to look now? | 23:30 |
| johnsom | Ok, yeah we couldn't wait any longer so hard coded a workaround. | 23:30 |
| johnsom | (that was months ago) | 23:31 |
| *** Julien-zte has quit IRC | 23:31 | |
| mtreinish | clarkb: but anyway I'm not sure how that stable ocata nova patch is hitting this failure mode. Because the change was never backported, especially to newton | 23:31 |
| mtreinish | ianw: there's no rush | 23:31 |
| mtreinish | johnsom: yeah, that's on me, sry. But you should be explicit about the image usage anyway, just for safety from the insanity of the devstack image code | 23:31 |
| mtreinish | clarkb: oh I know how it failed, oomichi just approved this: https://review.openstack.org/#/c/434352/ | 23:32 |
| mtreinish | ugh, let me push up a revert | 23:32 |
| mtreinish | and we should fast track it | 23:32 |
| openstackgerrit | Clint 'SpamapS' Byrum proposed openstack-infra/zuul feature/zuulv3: Add support for bwrap https://review.openstack.org/453851 | 23:32 |
| *** wolverineav has quit IRC | 23:33 | |
| mtreinish | ianw, clarkb: you might want to promote: https://review.openstack.org/#/c/460349/1 | 23:33 |
| *** wolverineav has joined #openstack-infra | 23:34 | |
| clarkb | oh thats on newton | 23:34 |
| clarkb | why would we change that in newton at this point? | 23:34 |
| *** zz_dimtruck is now known as dimtruck | 23:35 | |
| *** ramishra has joined #openstack-infra | 23:36 | |
| mtreinish | clarkb: this whole exercise was started because I want to remove all the amazon image handling code from tempest, (which has been deprecated for a while) to do that we need to only give qcow2 to tempest in devstack | 23:36 |
| mtreinish | it's this whole web of stuff I'm trying to unravel | 23:36 |
| clarkb | ah its beacuse tempest is single branch | 23:36 |
| mtreinish | yep | 23:37 |
| clarkb | ianw: do you want me to promote? | 23:37 |
| *** Sukhdev has quit IRC | 23:37 | |
| mtreinish | this was my fault for not properly WIPing the patches which shouldn't land yet | 23:37 |
| mtreinish | I'll do that now | 23:37 |
| ianw | clarkb: sure, if you're around | 23:37 |
| clarkb | ok doing it | 23:37 |
| clarkb | actually its quiet enoguh I may just need enqueue | 23:37 |
| ianw | yeah queue doesn't look ridiculous | 23:38 |
| *** wolverineav has quit IRC | 23:38 | |
| clarkb | and done | 23:39 |
| *** rlandy has quit IRC | 23:40 | |
| mtreinish | ok, I've abandonded and blocked the appropriate changes | 23:41 |
| mtreinish | I do need to make sure I fix this before the release, because otherwise grenade won't work when we branch | 23:41 |
| *** Sukhdev has joined #openstack-infra | 23:42 | |
| *** thorst_afk has joined #openstack-infra | 23:42 | |
| *** adrian_otto has quit IRC | 23:46 | |
| *** thorst_afk has quit IRC | 23:47 | |
| *** tosky has quit IRC | 23:50 | |
| kfox1111 | anyone familior with reno awake? | 23:51 |
| kfox1111 | is it ok to tweak a reno file from a different commit? | 23:51 |
| mtreinish | kfox1111: it depends on what you're trying to do. But if you just want to update the text in an existing note that's fine | 23:52 |
| *** stakeda has joined #openstack-infra | 23:52 | |
| mtreinish | it won't move the note across a release boundary though | 23:52 |
| kfox1111 | yeah. ok. thanks. | 23:53 |
| *** adrian_otto has joined #openstack-infra | 23:53 | |
| *** rfolco has joined #openstack-infra | 23:55 | |
| *** rfolco has quit IRC | 23:55 | |
| *** rfolco has joined #openstack-infra | 23:55 | |
| *** rfolco has quit IRC | 23:56 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/nodepool feature/zuulv3: Enforce cloud as a required config value https://review.openstack.org/460354 | 23:56 |
| *** rfolco has joined #openstack-infra | 23:56 | |
| *** rfolco has quit IRC | 23:57 | |
| *** rfolco has joined #openstack-infra | 23:57 | |
| *** rfolco has quit IRC | 23:57 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!