| mtreinish | fungi: I could have messed it up, IIRC I copied it from puppet-elastic_recheck though | 00:00 |
|---|---|---|
| *** rossella_s has quit IRC | 00:00 | |
| fungi | i think something has probably gone sideways with my ipv6 tunnel which is causing an interactive editor over ssh to puppetmaster to just hang at arbitrary points | 00:00 |
| pabelanger | jeblair: ++ | 00:00 |
| *** rossella_s has joined #openstack-infra | 00:00 | |
| jeblair | fungi: want me to type for you? | 00:00 |
| *** gildub has joined #openstack-infra | 00:00 | |
| mtreinish | fungi: yep: http://git.openstack.org/cgit/openstack-infra/puppet-elastic_recheck/tree/manifests/bot.pp#n70 | 00:00 |
| fungi | any change in the editor which shifts a significant number of characters pauses for 10+ seconds (sometimes indefinitely) | 00:00 |
| fungi | jeblair: i think i've got it now | 00:01 |
| mtreinish | and http://git.openstack.org/cgit/openstack-infra/puppet-germqtt/tree/manifests/server.pp#n91 | 00:01 |
| fungi | mtreinish: yeah, we're just inconsistent about how we do it across multiple modules and need to standardize at some point | 00:02 |
| *** sflanigan has quit IRC | 00:03 | |
| fungi | okay, so the reason to add the ip addresses along with the hostname is to prevent ssh from adding entries for the ip addresses on additional lines in known_hosts (which puppet will then keep resetting) | 00:03 |
| jeblair | fungi: i'm not seeing the inconsistency | 00:04 |
| jeblair | maybe i'm missing something | 00:04 |
| fungi | jeblair: i could have sworn in some places we templated the ip addresses so we didn't duplicate them all over the place | 00:04 |
| fungi | but also we'd be better off putting one entry in public hiera for this rather than individual copies in group and fqdn hiera files | 00:05 |
| jeblair | fungi: i see review, status, zuul, and zuul-mergers using the hiera key 'gerrit_ssh_rsa_pubkey_contents' which contains only "ssh-rsa <base64...> comment" | 00:05 |
| jeblair | fungi: and their uses in system-config templating into them with copied (and sometimes wrong, but that's beside the point here) values for the ip addresses/hostname | 00:06 |
| jeblair | modules/openstack_project/manifests/zuul_prod.pp: known_hosts_content => "review.openstack.org,104.130.159.134,2001:4800:7818:102:be76:4eff:fe05:9b12 ${gerrit_ssh_host_key}", | 00:06 |
| jeblair | like that | 00:06 |
| fungi | jeblair: yeah, we have other modules just stuffing all the content from hiera (like the one for the firehose server) | 00:06 |
| fungi | including hostname and (now) ip addresses | 00:07 |
| jeblair | fungi: that shouldn't be in the modules | 00:07 |
| fungi | actually the hiera entry when i found it didn't even have the hostname | 00:07 |
| fungi | but expected it | 00:07 |
| fungi | jeblair: agreed | 00:07 |
| fungi | which is what i meant by inconsistency | 00:07 |
| *** amitgandhinz has joined #openstack-infra | 00:08 | |
| fungi | according to mtreinish the elastic_recheck module does the same way, though i haven't looked yet | 00:08 |
| *** pahuang has joined #openstack-infra | 00:08 | |
| fungi | looking to see how to make them consistent with the ones you mentioned | 00:09 |
| jeblair | fungi: the er module expects both a host and a host key parameter, then concatenates them | 00:10 |
| jeblair | unfortunately, it also uses that parameter for other things | 00:11 |
| jeblair | so it will not support an 'ipv4,ipv6' formate | 00:11 |
| *** mtanino has quit IRC | 00:11 | |
| jeblair | i think it would be best to have both puppet-elastic_recheck and puppet-germqtt just take 'known_hosts_content' as a single string like the other modules | 00:12 |
| *** rlandy has quit IRC | 00:12 | |
| *** amitgandhinz has quit IRC | 00:12 | |
| fungi | also it looks like openssh would prefer the format to be more like... [review.openstack.org]:29418,[2001:4800:7819:103:be76:4eff:fe05:8525]:29418,[104.130.246.91]:29418 ssh-rsa AAA... | 00:12 |
| fungi | (port-specific entries) | 00:12 |
| jeblair | i think that's step 1, then step 2 is update system-config to configure those servers just like the others, then step 3 is to move the whole string blob into public hiera to make it easier to deal with | 00:13 |
| fungi | yep, i concur | 00:13 |
| fungi | working on that now | 00:13 |
| fungi | mtreinish: in the meantime, any content coming from the firehose now? | 00:13 |
| jeblair | fungi: so probably add a new argument "known_hosts_content=''" and if it has a value, have it supercede the old behavior, then later drop the old argument | 00:14 |
| fungi | yep, that should be safe | 00:14 |
| mtreinish | fungi: not that at I can see | 00:15 |
| *** esberglu has quit IRC | 00:15 | |
| pabelanger | jeblair: do you have a moment to confirm 349119 was what you were suggesting in your comments from 349119 | 00:16 |
| *** piet has quit IRC | 00:16 | |
| fungi | mtreinish: the last time i restarted germqtt (~12 minutes ago) the only thing it logged was... No handlers could be found for logger "gerrit.GerritWatcher" | 00:16 |
| mtreinish | fungi: hmm, not seeing anything | 00:17 |
| mtreinish | is there is anything in the mosquitto log? | 00:17 |
| mtreinish | maybe now it's getting events and not able to write them to mqtt | 00:18 |
| *** sflanigan has joined #openstack-infra | 00:18 | |
| fungi | mtreinish: it's logging new connections/closes and saving its in-memory database. no errors about events (or anything about events) | 00:20 |
| jeblair | pabelanger: yes, but see comment | 00:20 |
| fungi | mtreinish: though also gerrit show-connections isn't mentioning an active connection from germqtt | 00:20 |
| pabelanger | jeblair: sure, changing | 00:21 |
| mtreinish | fungi: hmm, the connections are probably me constantly reconnecting | 00:21 |
| mtreinish | are they all from a client id with sazabi in the name? | 00:21 |
| fungi | yep | 00:22 |
| mtreinish | fungi: hmm, that probably means germqtt isn't working | 00:22 |
| mtreinish | I really should add better logging to it | 00:22 |
| fungi | i'm systemd fumbling trying to find where/how the germqtt service is defined | 00:22 |
| mtreinish | as opposed to the none it has now :) | 00:22 |
| mtreinish | it's in /etc/systemd/system/germqtt.service | 00:22 |
| mtreinish | I had to write it by hand | 00:22 |
| fungi | aha, thanks | 00:22 |
| fungi | i've switched to running `/usr/local/bin/germqtt /etc/germqtt.conf --foreground` as the germqtt user to see if i get any better feedback | 00:23 |
| fungi | so far the only log entry is the one about the logger handler | 00:23 |
| fungi | No handlers could be found for logger "gerrit.GerritWatcher" | 00:24 |
| openstackgerrit | Paul Belanger proposed openstack-infra/nodepool: Include subnodes in updateStats function https://review.openstack.org/349119 | 00:24 |
| mtreinish | fungi: yeah it has 0 logging right now, so there isn't likely gonna be anything useful unless there is an error message | 00:25 |
| jeblair | pabelanger: +2 | 00:25 |
| pabelanger | jeblair: great, thanks for the help | 00:25 |
| fungi | mtreinish: it looks like it's connecting to gerrit over and over every ~5 seconds, according to tcpdump | 00:28 |
| fungi | i'll check gerrit's logs | 00:29 |
| mtreinish | fungi: did doing an ssh to review.o.o with the germqtt user show anything. I was normally able to debug issues doing that | 00:31 |
| *** thorst has joined #openstack-infra | 00:31 | |
| fungi | mtreinish: that worked. it's how i was diagnosing the known_hosts stuff | 00:31 |
| *** xarses has joined #openstack-infra | 00:32 | |
| *** rfolco_ has joined #openstack-infra | 00:32 | |
| *** rfolco_ has quit IRC | 00:32 | |
| fungi | `ssh -p 29418 germqtt@review.openstack.org gerrit stream-events` as the germqtt user on firehose01 works fine | 00:33 |
| fungi | and i see gerrit events | 00:33 |
| *** hichihara has joined #openstack-infra | 00:33 | |
| *** baoli has joined #openstack-infra | 00:33 | |
| mtreinish | hmm, I wonder what's up with gerritlib doing it then... | 00:33 |
| *** kaisers_ has joined #openstack-infra | 00:34 | |
| fungi | mtreinish: here's the gerrit section from germqtt.conf on the server http://paste.openstack.org/show/546942/ | 00:34 |
| jeblair | now might be a good time to add that logging | 00:34 |
| jeblair | logging.basicConfig when running --foreground might be sufficient | 00:35 |
| fungi | mtreinish: looks like the template says $gerrit_key belongs on that last line, and is empty | 00:35 |
| fungi | mtreinish: would that cause a problem? | 00:35 |
| mtreinish | fungi: yeah, that needs to point the public key file | 00:36 |
| mtreinish | did I make a typo? | 00:36 |
| fungi | gerrit_key doesn't seem to be a class parameter that i can find | 00:36 |
| mtreinish | fungi: oh, that's probably from an earlier rev when I thought it was gonna copy the file around as a param | 00:37 |
| mtreinish | until I looked at how e-r's puppet did it | 00:37 |
| fungi | hrm, another bug for you... germqtt doesn't handle sigint very well (or at all as far as i can tell) | 00:37 |
| mtreinish | so we should change that to /home/germqtt/.ssh/id_rsa | 00:37 |
| mtreinish | fungi: it's the gerritlib watcher thread | 00:38 |
| mtreinish | it waits for that to die | 00:38 |
| fungi | mtreinish: i wonder, shouldn't gerritlib default to using ~/.ssh/id_rsa anyway? | 00:38 |
| *** kaisers_ has quit IRC | 00:38 | |
| fungi | does it really need that specified? | 00:38 |
| *** thorst_ has joined #openstack-infra | 00:39 | |
| mtreinish | fungi: probably not, but that requires a germqtt change | 00:39 |
| mtreinish | since it is a required param right now | 00:40 |
| *** apetrich has quit IRC | 00:40 | |
| mtreinish | fungi: i'm getting events now \o/ | 00:40 |
| fungi | okay, looks like we have a live gerrit connection | 00:40 |
| fungi | yay! | 00:40 |
| fungi | that's after fixing the config manually | 00:41 |
| *** thorst has quit IRC | 00:41 | |
| *** thorst_ is now known as thorst | 00:41 | |
| *** karthik__ has quit IRC | 00:42 | |
| openstackgerrit | Matthew Treinish proposed openstack-infra/puppet-germqtt: Fix path for ssh key file https://review.openstack.org/350320 | 00:42 |
| mtreinish | fungi: ^^^ | 00:42 |
| *** apetrich has joined #openstack-infra | 00:42 | |
| mtreinish | fungi: although it doesn't look like the websocket port is working | 00:43 |
| mtreinish | oh, actually that's probably a client config issue. Lemme double check | 00:43 |
| fungi | i'll work on the ssh known_hosts cleanup across our puppet modules, but probably not tonight at this point | 00:43 |
| mtreinish | err, no it looks right | 00:44 |
| openstackgerrit | Brad Behle proposed openstack-infra/project-config: Make ovn-install-kuryr job voting https://review.openstack.org/350321 | 00:44 |
| fungi | still have some non-openstack things i need to get done with what's left of my evening | 00:44 |
| *** dprince has quit IRC | 00:44 | |
| *** baoli has quit IRC | 00:45 | |
| mtreinish | fungi: sure, thanks for the help. | 00:45 |
| *** sarob has quit IRC | 00:47 | |
| fungi | mtreinish: strangely, i still see connections being made to gerrit every 5 seconds according to tcpdump | 00:47 |
| fungi | which don't seem to be showing up in gerrit's ssh log | 00:47 |
| mtreinish | hmm, that's odd | 00:47 |
| fungi | filtering on just packets with "Flags [S]" there's one to 2001:4800:7819:103:be76:4eff:fe05:8525.29418 every 5 seconds | 00:48 |
| fungi | from various (increasing) ephemeral local ports | 00:49 |
| fungi | netstat shows a dozen or so of them in TIME_WAIT state | 00:50 |
| mtreinish | fungi: hmm, I don't know what could be causing that | 00:50 |
| fungi | you're still getting events? | 00:50 |
| *** _sarob has joined #openstack-infra | 00:50 | |
| mtreinish | fungi: also looks like the event stream shas stopped | 00:50 |
| fungi | oh, maybe this is connected | 00:51 |
| *** gongysh_ has joined #openstack-infra | 00:51 | |
| fungi | yeah, i see no established ssh connection in gerrit's connections list for the germqtt user now | 00:51 |
| fungi | so something caused it to die | 00:51 |
| fungi | aha, i bet puppet reconfigured it | 00:51 |
| fungi | the config is missing its key path again | 00:52 |
| mtreinish | fungi: haha | 00:52 |
| fungi | okay, i'll knock off for now. this should in theory correct itself | 00:52 |
| fungi | once yuor patch merges | 00:52 |
| fungi | your | 00:52 |
| *** hichihara has quit IRC | 00:53 | |
| *** hichihara has joined #openstack-infra | 00:53 | |
| mtreinish | fungi: heh, hopefully at least :) | 00:53 |
| *** sarob has joined #openstack-infra | 00:54 | |
| fungi | jeblair: on the list of inconsistent known_hosts management for gerrit access, proposal.slave has no ~jenkins/.ssh/known_hosts at all | 00:54 |
| *** _sarob has quit IRC | 00:55 | |
| *** karthik__ has joined #openstack-infra | 00:56 | |
| fungi | (nor does signing01.ci because i forgot to add it until now) | 00:57 |
| fungi | ohhhh | 00:57 |
| fungi | this is unfortunate | 00:57 |
| *** sdake has quit IRC | 00:57 | |
| fungi | ~jenkins/.ssh/config on proposal.slave has UserKnownHostsFile=/dev/null and StrictHostKeyChecking=no | 00:58 |
| fungi | i'll look into this in more detail in the morning | 00:58 |
| fungi | same on signing01.ci | 00:58 |
| *** sarob has quit IRC | 00:58 | |
| *** sdake has joined #openstack-infra | 00:59 | |
| fungi | looks like puppet-jenkins sets it | 01:00 |
| fungi | specifically the jenkins::jenkinsuser class | 01:00 |
| fungi | guess i'll work on that tomorrow too | 01:00 |
| fungi | night all | 01:01 |
| jeblair | fungi: you may need to ask <james.blair@rackspace.com> why that is | 01:01 |
| fungi | hah | 01:01 |
| jeblair | fungi: good luck getting in touch with that guy! | 01:01 |
| fungi | git blame to teh rescue! | 01:01 |
| fungi | something tells me that e-mail address no work good | 01:01 |
| jeblair | that commit includes words like "orchestra" "tarmac" "lvm snapshot" and "cobbler" | 01:02 |
| jeblair | that's like 4 blasts from the past | 01:02 |
| fungi | now i'm gonna have nighthmares | 01:02 |
| *** tphummel has quit IRC | 01:02 | |
| fungi | nightmares | 01:02 |
| *** tphummel has joined #openstack-infra | 01:03 | |
| fungi | thankfully all things you had fixed by the time i came onto the scene | 01:03 |
| fungi | (for various definitions of "fixed" anyway) | 01:03 |
| jeblair | to be fair, i *created* "lvm snapshot", but also got rid of it :) | 01:03 |
| *** ramishra has quit IRC | 01:04 | |
| jeblair | fungi: the upshot i see from this is that there is that there is that the reasoning that led to "Remove known_hosts file, add .ssh/config file that ignores known_hosts." from that commit is likely no longer relevant, and i think it should be safe to alter our approach. | 01:04 |
| fungi | i'll guessing this was around the same timeframe as the gpg signing key i just removed from jenkinsuser | 01:05 |
| fungi | for the packages in our ppa | 01:05 |
| *** flyingtt has quit IRC | 01:05 | |
| jeblair | indeed :) | 01:05 |
| jeblair | also, that was a fantastic sentence i just constructed. i don't think i can top that, so i think i'll go do something else now. | 01:06 |
| fungi | wondering if we have any jobs which have grown a dependence on this configuration now though | 01:06 |
| fungi | like, that ssh to something on localhost as the jenkins user | 01:06 |
| fungi | and are blissfully ignorant of any need to handle host keys as a result | 01:07 |
| *** karthik__ has quit IRC | 01:07 | |
| fungi | so if we just rip it out, we could see some unfortunateness in some jobs, but i can't think of a way at the moment to find and fix those without breaking them | 01:08 |
| *** tphummel has quit IRC | 01:08 | |
| *** pvaneck has quit IRC | 01:08 | |
| fungi | or i could just make it conditional and only apply to single-use nodes | 01:08 |
| fungi | so we can do proper host key management on our long-lived nodes | 01:08 |
| *** amitgandhinz has joined #openstack-infra | 01:09 | |
| fungi | anyway, topics for tomorrow. really disappearing now. night! | 01:09 |
| *** zhurong has joined #openstack-infra | 01:10 | |
| *** psilvad has joined #openstack-infra | 01:10 | |
| *** Sukhdev has quit IRC | 01:12 | |
| *** shashank_hegde has quit IRC | 01:12 | |
| *** Apoorva has quit IRC | 01:12 | |
| *** amitgandhinz has quit IRC | 01:13 | |
| *** mriedem has quit IRC | 01:19 | |
| *** karthik__ has joined #openstack-infra | 01:19 | |
| *** akshai has joined #openstack-infra | 01:23 | |
| *** Apoorva has joined #openstack-infra | 01:23 | |
| *** dimtruck is now known as zz_dimtruck | 01:23 | |
| *** tqtran has quit IRC | 01:23 | |
| *** hichihara has quit IRC | 01:24 | |
| *** hichihara has joined #openstack-infra | 01:24 | |
| *** cody-somerville has joined #openstack-infra | 01:26 | |
| openstackgerrit | Craige McWhirter proposed openstack-infra/puppet-phabricator: Deleting files surplus to this module. https://review.openstack.org/350330 | 01:26 |
| *** akshai has quit IRC | 01:35 | |
| openstackgerrit | Craige McWhirter proposed openstack-infra/puppet-phabricator: Patches Required to Deliver Pholio https://review.openstack.org/342481 | 01:36 |
| *** moravec has joined #openstack-infra | 01:40 | |
| *** Apoorva_ has joined #openstack-infra | 01:40 | |
| *** sdake has quit IRC | 01:44 | |
| *** Apoorva has quit IRC | 01:44 | |
| *** Apoorva_ has quit IRC | 01:44 | |
| prometheanfire | gate stalled? | 01:46 |
| *** thorst has quit IRC | 01:47 | |
| *** thorst has joined #openstack-infra | 01:48 | |
| *** baoli has joined #openstack-infra | 01:49 | |
| *** itisha has quit IRC | 01:50 | |
| *** yanyanhu has joined #openstack-infra | 01:52 | |
| *** karthik__ has quit IRC | 01:53 | |
| *** cody-somerville has quit IRC | 01:55 | |
| *** pt_15 has quit IRC | 01:56 | |
| *** thorst has quit IRC | 01:56 | |
| *** zz_dimtruck is now known as dimtruck | 01:58 | |
| *** cody-somerville has joined #openstack-infra | 01:59 | |
| *** shashank_hegde has joined #openstack-infra | 02:00 | |
| *** amitgandhinz has joined #openstack-infra | 02:09 | |
| *** tphummel has joined #openstack-infra | 02:10 | |
| *** psilvad has quit IRC | 02:11 | |
| *** pahuang has quit IRC | 02:14 | |
| *** amitgandhinz has quit IRC | 02:14 | |
| *** gildub has quit IRC | 02:17 | |
| openstackgerrit | Dean Troyer proposed openstack/os-client-config: Fix precedence for pass-in options https://review.openstack.org/350346 | 02:18 |
| *** coolsvap has joined #openstack-infra | 02:19 | |
| *** ramishra has joined #openstack-infra | 02:21 | |
| *** kaisers_ has joined #openstack-infra | 02:23 | |
| *** gouthamr has joined #openstack-infra | 02:24 | |
| *** ramishra has quit IRC | 02:26 | |
| *** gouthamr_ has joined #openstack-infra | 02:27 | |
| *** kaisers_ has quit IRC | 02:27 | |
| *** ramishra has joined #openstack-infra | 02:28 | |
| *** yamahata has quit IRC | 02:28 | |
| *** elarson has quit IRC | 02:29 | |
| *** gouthamr has quit IRC | 02:30 | |
| *** reed_ has joined #openstack-infra | 02:34 | |
| *** reed_ has quit IRC | 02:36 | |
| *** adrian_otto has joined #openstack-infra | 02:39 | |
| *** markvoelker has joined #openstack-infra | 02:39 | |
| *** zhurong has quit IRC | 02:39 | |
| *** zhurong has joined #openstack-infra | 02:40 | |
| openstackgerrit | Craige McWhirter proposed openstack-infra/puppet-phabricator: Patches Required to Deliver Pholio https://review.openstack.org/342481 | 02:42 |
| *** tphummel has quit IRC | 02:42 | |
| *** tphummel has joined #openstack-infra | 02:43 | |
| *** camunoz has quit IRC | 02:45 | |
| *** sflanigan has quit IRC | 02:46 | |
| *** yuanying has quit IRC | 02:48 | |
| *** piet has joined #openstack-infra | 02:48 | |
| adrian_otto | Does anyone know how to upload image files to the OpenStack Wiki? I have done it in the past, but now I can't figure out where the file upload feature is. Was it taken away? | 02:50 |
| loquacities | adrian_otto: i believe it was removed after the Great Spam Attack | 02:51 |
| adrian_otto | any chance it could be temporarily re-enabled? | 02:51 |
| *** thorst has joined #openstack-infra | 02:54 | |
| openstackgerrit | Craige McWhirter proposed openstack-infra/puppet-phabricator: Patches Required to Deliver Pholio https://review.openstack.org/342481 | 02:56 |
| *** sflanigan has joined #openstack-infra | 02:58 | |
| *** hemanthm|afk is now known as hemanthm | 02:59 | |
| *** piet has quit IRC | 03:01 | |
| *** thorst has quit IRC | 03:02 | |
| *** oanson has joined #openstack-infra | 03:03 | |
| *** Srinu has joined #openstack-infra | 03:05 | |
| lifeless | fungi: the stock tox.ini's that Nakato put together all use an environment variable | 03:07 |
| lifeless | fungi: so no change should be needed there - we already down the constraints file locally in python jobs, its just not edited at the moment | 03:07 |
| openstackgerrit | Steve Baker proposed openstack-infra/shade: Depend on python-heatclient>=1.0.0 https://review.openstack.org/350360 | 03:08 |
| *** docaedo has quit IRC | 03:10 | |
| *** signed8bit_Zzz is now known as signed8bit | 03:10 | |
| *** amitgandhinz has joined #openstack-infra | 03:10 | |
| *** signed8bit has quit IRC | 03:10 | |
| *** oanson has quit IRC | 03:11 | |
| *** amitgandhinz has quit IRC | 03:15 | |
| *** gildub has joined #openstack-infra | 03:15 | |
| *** adrian_otto has quit IRC | 03:18 | |
| *** docaedo has joined #openstack-infra | 03:24 | |
| *** r-mibu has quit IRC | 03:24 | |
| *** r-mibu has joined #openstack-infra | 03:24 | |
| *** shashank_hegde has quit IRC | 03:24 | |
| *** dimtruck is now known as zz_dimtruck | 03:26 | |
| *** zz_dimtruck is now known as dimtruck | 03:26 | |
| *** Nakato has quit IRC | 03:27 | |
| *** gouthamr_ has quit IRC | 03:29 | |
| *** camunoz has joined #openstack-infra | 03:32 | |
| *** Nakato has joined #openstack-infra | 03:32 | |
| *** yamamoto_ has joined #openstack-infra | 03:33 | |
| *** tphummel has quit IRC | 03:35 | |
| *** roxanaghe has joined #openstack-infra | 03:35 | |
| *** dimtruck is now known as zz_dimtruck | 03:35 | |
| openstackgerrit | Merged openstack/os-client-config: Update citycloud to list new regions https://review.openstack.org/339990 | 03:38 |
| *** EricGonczer_ has quit IRC | 03:43 | |
| *** yuanying has joined #openstack-infra | 03:47 | |
| *** ayoung has quit IRC | 03:48 | |
| *** yuanying has quit IRC | 03:49 | |
| *** yuanying has joined #openstack-infra | 03:49 | |
| *** sflanigan has quit IRC | 03:49 | |
| Srinu | Hi. In jenkins after completion of Job(whether success/failure) how can we stop not killing the slave.jar process running on the slaves | 03:49 |
| *** tphummel has joined #openstack-infra | 03:50 | |
| *** sdake has joined #openstack-infra | 03:50 | |
| *** sdake_ has joined #openstack-infra | 03:54 | |
| *** markvoelker has quit IRC | 03:55 | |
| *** sdake has quit IRC | 03:58 | |
| *** thorst has joined #openstack-infra | 03:59 | |
| *** rossella_s has quit IRC | 04:00 | |
| *** rossella_s has joined #openstack-infra | 04:00 | |
| *** tphummel has quit IRC | 04:01 | |
| openstackgerrit | Merged openstack-infra/puppet-mosquitto: Enable real logging https://review.openstack.org/350249 | 04:02 |
| *** sdake_ has quit IRC | 04:05 | |
| *** sflanigan has joined #openstack-infra | 04:06 | |
| *** sflanigan has joined #openstack-infra | 04:06 | |
| *** thorst has quit IRC | 04:06 | |
| *** zhurong has quit IRC | 04:07 | |
| *** links has joined #openstack-infra | 04:10 | |
| *** amitgandhinz has joined #openstack-infra | 04:11 | |
| *** zhurong has joined #openstack-infra | 04:12 | |
| Srinu | i am using two slaves for running 2 jobs and when one job finished first while other job not finished yet while deletion of nodepool vm of completed job,2 slave java process is killed Slave JVM has terminated. did any one face this issue | 04:13 |
| *** Sukhdev has joined #openstack-infra | 04:13 | |
| *** amitgandhinz has quit IRC | 04:16 | |
| openstackgerrit | Craige McWhirter proposed openstack-infra/puppet-phabricator: Configure HTTPD and HTTPS certificates https://review.openstack.org/350370 | 04:16 |
| *** amotoki has joined #openstack-infra | 04:20 | |
| openstackgerrit | Craige McWhirter proposed openstack-infra/puppet-phabricator: Patches Required to Deliver Pholio https://review.openstack.org/342481 | 04:20 |
| *** shashank_hegde has joined #openstack-infra | 04:20 | |
| *** kaisers_ has joined #openstack-infra | 04:23 | |
| *** kaisers_ has quit IRC | 04:23 | |
| openstackgerrit | Craige McWhirter proposed openstack-infra/puppet-phabricator: Patches Required to Deliver Pholio https://review.openstack.org/342481 | 04:27 |
| *** shashank_hegde has quit IRC | 04:29 | |
| *** markvoelker has joined #openstack-infra | 04:33 | |
| openstackgerrit | Craige McWhirter proposed openstack-infra/puppet-phabricator: Add MySQl & Other Configuration Options to Pholio https://review.openstack.org/350374 | 04:34 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/bindep: Add .bindeprc as additional filename https://review.openstack.org/350184 | 04:36 |
| *** markvoelker has quit IRC | 04:39 | |
| openstackgerrit | Anusha Ramineni proposed openstack-infra/project-config: Make python35 jobs voting on congress projects https://review.openstack.org/350375 | 04:42 |
| *** furlongm has quit IRC | 04:43 | |
| *** furlongm_ has joined #openstack-infra | 04:43 | |
| *** gildub has quit IRC | 04:45 | |
| *** shashank_hegde has joined #openstack-infra | 04:46 | |
| *** shashank_hegde has quit IRC | 04:51 | |
| *** psachin has joined #openstack-infra | 04:54 | |
| openstackgerrit | Merged openstack-infra/zuul: Remove build descriptions https://review.openstack.org/347015 | 04:58 |
| *** rfolco has joined #openstack-infra | 05:02 | |
| *** julim has quit IRC | 05:02 | |
| openstackgerrit | Merged openstack-infra/zuul: Remove job registration check https://review.openstack.org/346236 | 05:02 |
| openstackgerrit | Merged openstack-infra/zuul: Re-enable test_jobs_launched https://review.openstack.org/346237 | 05:02 |
| openstackgerrit | Merged openstack-infra/zuul: Re-enable test_initial_pipeline_gauges https://review.openstack.org/347070 | 05:03 |
| openstackgerrit | Merged openstack-infra/zuul: Move test_cloner out of its own subdirectory https://review.openstack.org/347071 | 05:03 |
| *** jaosorior has joined #openstack-infra | 05:03 | |
| openstackgerrit | Merged openstack-infra/zuul: Add utility program to print test fixture configuration https://review.openstack.org/347072 | 05:03 |
| *** pahuang has joined #openstack-infra | 05:03 | |
| openstackgerrit | Merged openstack-infra/zuul: Use RecordingLaunchServer to run all tests https://review.openstack.org/350165 | 05:03 |
| *** thorst has joined #openstack-infra | 05:04 | |
| *** roxanaghe has quit IRC | 05:05 | |
| *** furlongm_ has quit IRC | 05:05 | |
| *** furlongm_ has joined #openstack-infra | 05:06 | |
| *** yamahata has joined #openstack-infra | 05:06 | |
| *** rfolco has quit IRC | 05:06 | |
| *** ihrachys has joined #openstack-infra | 05:08 | |
| *** karthikp_ has joined #openstack-infra | 05:09 | |
| *** rbuzatu has joined #openstack-infra | 05:09 | |
| *** tphummel has joined #openstack-infra | 05:11 | |
| *** thorst has quit IRC | 05:11 | |
| *** amitgandhinz has joined #openstack-infra | 05:11 | |
| *** e0ne has joined #openstack-infra | 05:12 | |
| *** e0ne has quit IRC | 05:15 | |
| *** amitgandhinz has quit IRC | 05:16 | |
| *** ihrachys has quit IRC | 05:17 | |
| *** yamahata has quit IRC | 05:19 | |
| *** aeng has joined #openstack-infra | 05:22 | |
| *** csomerville has joined #openstack-infra | 05:24 | |
| *** oanson has joined #openstack-infra | 05:26 | |
| *** cody-somerville has quit IRC | 05:27 | |
| *** armax has quit IRC | 05:28 | |
| *** markvoelker has joined #openstack-infra | 05:28 | |
| *** rcernin has joined #openstack-infra | 05:28 | |
| *** markvoelker has quit IRC | 05:33 | |
| *** aeng has quit IRC | 05:39 | |
| *** ihrachys has joined #openstack-infra | 05:46 | |
| *** shashank_hegde has joined #openstack-infra | 05:50 | |
| openstackgerrit | Merged openstack-infra/system-config: Don't override the mosquitto log path https://review.openstack.org/350295 | 05:52 |
| *** Sukhdev has quit IRC | 05:55 | |
| *** julim has joined #openstack-infra | 05:57 | |
| openstackgerrit | Merged openstack-infra/system-config: Enable ntpdate service for centos-7 https://review.openstack.org/349692 | 05:58 |
| *** ihrachys has quit IRC | 05:58 | |
| *** gildub has joined #openstack-infra | 05:59 | |
| *** sflanigan has quit IRC | 06:02 | |
| *** timl0101 has quit IRC | 06:06 | |
| *** sdake has joined #openstack-infra | 06:07 | |
| *** sdake has quit IRC | 06:07 | |
| *** sdake has joined #openstack-infra | 06:07 | |
| *** timl0101 has joined #openstack-infra | 06:07 | |
| *** bhavik has joined #openstack-infra | 06:08 | |
| *** javeriak has joined #openstack-infra | 06:09 | |
| *** thorst has joined #openstack-infra | 06:09 | |
| *** amitgandhinz has joined #openstack-infra | 06:12 | |
| *** sflanigan has joined #openstack-infra | 06:15 | |
| *** sflanigan has joined #openstack-infra | 06:15 | |
| *** tphummel has quit IRC | 06:15 | |
| *** thorst has quit IRC | 06:16 | |
| *** amitgandhinz has quit IRC | 06:17 | |
| *** mikelk has joined #openstack-infra | 06:17 | |
| *** rfolco has joined #openstack-infra | 06:18 | |
| *** mhickey has joined #openstack-infra | 06:19 | |
| *** tqtran has joined #openstack-infra | 06:21 | |
| *** tphummel has joined #openstack-infra | 06:22 | |
| *** markvoelker has joined #openstack-infra | 06:22 | |
| *** florianf has joined #openstack-infra | 06:23 | |
| *** sdake has quit IRC | 06:23 | |
| *** rfolco has quit IRC | 06:24 | |
| *** tqtran has quit IRC | 06:25 | |
| *** markvoelker has quit IRC | 06:26 | |
| *** flepied has joined #openstack-infra | 06:27 | |
| openstackgerrit | Merged openstack-infra/puppet-germqtt: Fix path for ssh key file https://review.openstack.org/350320 | 06:30 |
| *** sarob has joined #openstack-infra | 06:31 | |
| *** sarob has quit IRC | 06:36 | |
| *** pgadiya has joined #openstack-infra | 06:36 | |
| yolanda | good morning | 06:40 |
| openstackgerrit | Csaba Henk proposed openstack-infra/project-config: templatize Manila GlusterFS jobs https://review.openstack.org/350120 | 06:42 |
| openstackgerrit | Craige McWhirter proposed openstack-infra/infra-specs: Initial commit of the Pholio spec https://review.openstack.org/340641 | 06:45 |
| *** tesseract- has joined #openstack-infra | 06:45 | |
| AJaeger | good morning, yolanda ! | 06:46 |
| *** gildub has quit IRC | 06:49 | |
| *** thorst has joined #openstack-infra | 06:55 | |
| *** thorst has quit IRC | 07:03 | |
| *** karthikp_ has quit IRC | 07:03 | |
| *** pahuang has quit IRC | 07:04 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config: Add py27, py35 jobs for openstack-doc-tools https://review.openstack.org/350397 | 07:04 |
| openstackgerrit | Merged openstack-infra/project-config: Add integration test job for Senlin https://review.openstack.org/347160 | 07:05 |
| openstackgerrit | Merged openstack-infra/project-config: Add tarball publisher to tripleo-ci for images https://review.openstack.org/350153 | 07:09 |
| rcarrillocruz | fungi , mtreinish : hiya. perplexed, so changing the log dest allows to bind ports < 1024 ? | 07:12 |
| rcarrillocruz | that's a veeery odd bug | 07:12 |
| *** amitgandhinz has joined #openstack-infra | 07:13 | |
| *** ihrachys has joined #openstack-infra | 07:14 | |
| *** jaosorior is now known as jaosorior_brb | 07:15 | |
| *** markvoelker has joined #openstack-infra | 07:16 | |
| *** e0ne has joined #openstack-infra | 07:16 | |
| *** pahuang has joined #openstack-infra | 07:17 | |
| *** berendt has joined #openstack-infra | 07:17 | |
| *** amitgandhinz has quit IRC | 07:18 | |
| *** berendt has quit IRC | 07:19 | |
| *** pcaruana has joined #openstack-infra | 07:21 | |
| *** markvoelker has quit IRC | 07:21 | |
| openstackgerrit | Csaba Henk proposed openstack-infra/project-config: templatize Manila GlusterFS jobs https://review.openstack.org/350120 | 07:22 |
| *** links has quit IRC | 07:23 | |
| *** Na3iL has joined #openstack-infra | 07:24 | |
| *** jlanoux has joined #openstack-infra | 07:24 | |
| *** jlanoux has quit IRC | 07:24 | |
| *** jlanoux has joined #openstack-infra | 07:26 | |
| *** ekarlso has quit IRC | 07:29 | |
| openstackgerrit | Merged openstack-infra/system-config: Fix resource references should now be capitalized warnings https://review.openstack.org/350015 | 07:30 |
| *** jpich has joined #openstack-infra | 07:35 | |
| *** links has joined #openstack-infra | 07:36 | |
| *** ekarlso has joined #openstack-infra | 07:37 | |
| *** claudiub has joined #openstack-infra | 07:37 | |
| openstackgerrit | Merged openstack-infra/project-config: telemetry: Don't enable panko on old branch https://review.openstack.org/344836 | 07:38 |
| *** shashank_hegde has quit IRC | 07:38 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/bindep: Add .bindeprc as additional filename https://review.openstack.org/350184 | 07:38 |
| *** javeriak has quit IRC | 07:38 | |
| openstackgerrit | Merged openstack-infra/project-config: Run basic tests in check and gate https://review.openstack.org/341902 | 07:38 |
| *** furlongm_ has quit IRC | 07:38 | |
| openstackgerrit | Merged openstack-infra/project-config: Update networking-generic-switch-dsvm-tempest job https://review.openstack.org/350062 | 07:38 |
| *** tphummel has quit IRC | 07:40 | |
| *** vincentll has joined #openstack-infra | 07:40 | |
| *** furlongm_ has joined #openstack-infra | 07:42 | |
| openstackgerrit | Merged openstack-infra/project-config: Add Ironic multinode job https://review.openstack.org/332490 | 07:44 |
| openstackgerrit | yolanda.robla proposed openstack-infra/puppet-jeepyb: Require packages before installing jeepyb https://review.openstack.org/327707 | 07:52 |
| *** Goneri has joined #openstack-infra | 07:54 | |
| openstackgerrit | Merged openstack-infra/project-config: Make python35 jobs voting on congress projects https://review.openstack.org/350375 | 07:55 |
| *** Goneri has quit IRC | 07:56 | |
| *** Goneri has joined #openstack-infra | 07:56 | |
| *** daemontool has joined #openstack-infra | 07:57 | |
| *** shardy has joined #openstack-infra | 07:59 | |
| *** jistr is now known as jistr|training | 07:59 | |
| *** zzzeek has quit IRC | 08:00 | |
| *** amotoki has quit IRC | 08:00 | |
| *** thorst has joined #openstack-infra | 08:01 | |
| openstackgerrit | Ricardo Carrillo Cruz proposed openstack-infra/shade: Add update_server method https://review.openstack.org/345630 | 08:02 |
| *** zzzeek has joined #openstack-infra | 08:02 | |
| *** amotoki has joined #openstack-infra | 08:04 | |
| *** amotoki has quit IRC | 08:05 | |
| *** atom_ has joined #openstack-infra | 08:06 | |
| *** coolsvap is now known as coolsvap_ | 08:08 | |
| *** thorst has quit IRC | 08:08 | |
| *** markvoelker has joined #openstack-infra | 08:10 | |
| *** Hal has quit IRC | 08:12 | |
| *** jaosorior_brb has quit IRC | 08:12 | |
| *** jaosorior_brb has joined #openstack-infra | 08:13 | |
| *** amitgandhinz has joined #openstack-infra | 08:14 | |
| *** markvoelker has quit IRC | 08:14 | |
| *** amotoki has joined #openstack-infra | 08:17 | |
| *** Hal has joined #openstack-infra | 08:17 | |
| *** amotoki_ has joined #openstack-infra | 08:18 | |
| *** Ravikiran_K has joined #openstack-infra | 08:18 | |
| *** amitgandhinz has quit IRC | 08:18 | |
| *** amotoki has quit IRC | 08:21 | |
| *** jaosorior_brb is now known as jaosorior | 08:23 | |
| *** tqtran has joined #openstack-infra | 08:23 | |
| *** tqtran has quit IRC | 08:27 | |
| *** baoli has quit IRC | 08:27 | |
| *** amotoki_ has quit IRC | 08:29 | |
| *** vincentll has quit IRC | 08:31 | |
| *** yaume has joined #openstack-infra | 08:35 | |
| *** mouli has joined #openstack-infra | 08:37 | |
| mouli | Hi All | 08:37 |
| mouli | I am facing the issue in updating the contact information in gerrit | 08:38 |
| mouli | In browser I tried to update the contact information and it returned with Code-review error cannot store contact information | 08:38 |
| mouli | your help will be appreciated | 08:38 |
| AJaeger | mouli, you need to be a "Foundation member" to be able to contribute | 08:41 |
| AJaeger | This needs your gerrit preferred e-mail address to match a primary | 08:41 |
| AJaeger | e-mail address for a foundation individual member account. | 08:41 |
| AJaeger | mouli: If you already followed the instructions at http://docs.openstack.org/infra/manual/developers.html#account-setup completely and in order, and still get that, see https://ask.openstack.org/question/56720 for additional troubleshooting tips. | 08:42 |
| openstackgerrit | Witold Bedyk proposed openstack-infra/project-config: Disable services for tempest gate https://review.openstack.org/350454 | 08:43 |
| *** electrofelix has joined #openstack-infra | 08:43 | |
| *** amotoki has joined #openstack-infra | 08:45 | |
| *** vincentll has joined #openstack-infra | 08:46 | |
| *** priteau has joined #openstack-infra | 08:48 | |
| *** jistr|training is now known as jistr | 08:49 | |
| *** mhickey has quit IRC | 08:51 | |
| *** atom_ has quit IRC | 08:52 | |
| *** amotoki has quit IRC | 08:53 | |
| *** jpr has joined #openstack-infra | 08:53 | |
| mouli | Thanks AJaeger, issue fixed | 08:53 |
| *** eliqiao has quit IRC | 08:54 | |
| *** ihrachys has quit IRC | 08:54 | |
| *** eliqiao has joined #openstack-infra | 08:55 | |
| *** zakora_ has joined #openstack-infra | 08:56 | |
| *** zakora_ has quit IRC | 08:56 | |
| *** hichihara has quit IRC | 08:57 | |
| *** vincentll has quit IRC | 08:59 | |
| *** binbincong has joined #openstack-infra | 08:59 | |
| *** mikelk has quit IRC | 09:00 | |
| *** amotoki has joined #openstack-infra | 09:02 | |
| *** shardy is now known as shardy_mtg | 09:02 | |
| *** asettle has joined #openstack-infra | 09:02 | |
| *** amitkqed has quit IRC | 09:04 | |
| *** markvoelker has joined #openstack-infra | 09:04 | |
| *** amitkqed has joined #openstack-infra | 09:04 | |
| AJaeger | glad to hear, mouli ! Happy contributing! | 09:05 |
| *** thorst has joined #openstack-infra | 09:06 | |
| *** Srinu has quit IRC | 09:08 | |
| *** markvoelker has quit IRC | 09:09 | |
| *** sambetts|afk is now known as sambetts | 09:11 | |
| *** mikelk has joined #openstack-infra | 09:11 | |
| *** mrmartin has joined #openstack-infra | 09:11 | |
| *** mouli has quit IRC | 09:11 | |
| *** thorst has quit IRC | 09:13 | |
| *** vincentll has joined #openstack-infra | 09:14 | |
| *** amitgandhinz has joined #openstack-infra | 09:15 | |
| *** amitgandhinz has quit IRC | 09:19 | |
| *** null0 has joined #openstack-infra | 09:23 | |
| *** dtantsur|afk is now known as dtantsur | 09:24 | |
| openstackgerrit | Martin Mágr proposed openstack-infra/tripleo-ci: Add opstools candidate repo https://review.openstack.org/325095 | 09:24 |
| openstackgerrit | yolanda.robla proposed openstack-infra/puppet-jeepyb: Require packages before installing jeepyb https://review.openstack.org/327707 | 09:30 |
| *** null0 has quit IRC | 09:33 | |
| *** john-davidge has joined #openstack-infra | 09:36 | |
| *** furlongm_ has quit IRC | 09:37 | |
| *** john-dav_ has joined #openstack-infra | 09:37 | |
| *** furlongm_ has joined #openstack-infra | 09:37 | |
| openstackgerrit | Ricardo Carrillo Cruz proposed openstack-infra/shade: Add update_server method https://review.openstack.org/345630 | 09:40 |
| *** electrofelix has quit IRC | 09:40 | |
| *** electrofelix has joined #openstack-infra | 09:40 | |
| *** john-davidge has quit IRC | 09:41 | |
| *** yamamoto_ has quit IRC | 09:41 | |
| *** skolekonov has joined #openstack-infra | 09:42 | |
| *** skolekonov has quit IRC | 09:42 | |
| *** evgenyl has quit IRC | 09:42 | |
| *** tlbr has joined #openstack-infra | 09:43 | |
| *** shardy_mtg is now known as shardy | 09:43 | |
| *** rguillebert has quit IRC | 09:43 | |
| *** evgenyl has joined #openstack-infra | 09:44 | |
| *** kambiz has joined #openstack-infra | 09:44 | |
| *** rguillebert has joined #openstack-infra | 09:44 | |
| *** simonmcc has quit IRC | 09:44 | |
| *** sflanigan has quit IRC | 09:46 | |
| *** sflanigan has joined #openstack-infra | 09:46 | |
| *** Qiming has quit IRC | 09:46 | |
| *** simonmcc has joined #openstack-infra | 09:47 | |
| *** hughhalf has quit IRC | 09:47 | |
| *** Qiming has joined #openstack-infra | 09:50 | |
| *** javeriak has joined #openstack-infra | 09:51 | |
| *** hughhalf has joined #openstack-infra | 09:52 | |
| *** yamahata has joined #openstack-infra | 09:56 | |
| *** dirk has quit IRC | 09:57 | |
| *** dirk has joined #openstack-infra | 09:57 | |
| *** Jaison has joined #openstack-infra | 09:58 | |
| *** itsuugo has quit IRC | 09:58 | |
| *** markvoelker has joined #openstack-infra | 09:58 | |
| *** links has quit IRC | 09:59 | |
| *** itsuugo has joined #openstack-infra | 10:00 | |
| *** nmagnezi has joined #openstack-infra | 10:01 | |
| *** markvoelker has quit IRC | 10:04 | |
| *** Goneri has quit IRC | 10:08 | |
| *** _degorenko|afk is now known as degorenko | 10:10 | |
| *** chem has joined #openstack-infra | 10:10 | |
| *** amit213 has quit IRC | 10:10 | |
| *** yamahata has quit IRC | 10:11 | |
| *** thorst has joined #openstack-infra | 10:11 | |
| openstackgerrit | Rob Cresswell proposed openstack-infra/project-config: Add nv test job for Django 1.10 in Django OS Auth https://review.openstack.org/350113 | 10:12 |
| *** amit213 has joined #openstack-infra | 10:12 | |
| *** john-dav_ is now known as john-davidge | 10:12 | |
| *** Qiming has quit IRC | 10:12 | |
| *** zhurong has quit IRC | 10:13 | |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack-infra/project-config: Add common testing repo to OpenStack-Ansible https://review.openstack.org/350503 | 10:14 |
| *** serverascode has quit IRC | 10:15 | |
| *** amitgandhinz has joined #openstack-infra | 10:15 | |
| *** bastelfreak has quit IRC | 10:16 | |
| *** tdasilva has quit IRC | 10:16 | |
| *** serverascode has joined #openstack-infra | 10:16 | |
| *** yamamoto has joined #openstack-infra | 10:17 | |
| *** Qiming has joined #openstack-infra | 10:17 | |
| *** thorst has quit IRC | 10:18 | |
| *** bastelfreak has joined #openstack-infra | 10:18 | |
| *** amitgandhinz has quit IRC | 10:20 | |
| *** rfolco has joined #openstack-infra | 10:20 | |
| *** javeriak has quit IRC | 10:21 | |
| *** yamamoto has quit IRC | 10:22 | |
| *** yamamoto has joined #openstack-infra | 10:23 | |
| *** Goneri has joined #openstack-infra | 10:24 | |
| *** tosky has joined #openstack-infra | 10:25 | |
| *** rfolco has quit IRC | 10:25 | |
| *** asettle has quit IRC | 10:27 | |
| *** asettle has joined #openstack-infra | 10:27 | |
| *** yamamoto has quit IRC | 10:28 | |
| *** amotoki has quit IRC | 10:30 | |
| *** rodrigods has quit IRC | 10:30 | |
| *** rodrigods has joined #openstack-infra | 10:30 | |
| *** mhickey has joined #openstack-infra | 10:36 | |
| *** tdasilva has joined #openstack-infra | 10:38 | |
| *** amotoki has joined #openstack-infra | 10:39 | |
| *** tosky has quit IRC | 10:39 | |
| *** tosky has joined #openstack-infra | 10:39 | |
| *** tosky has joined #openstack-infra | 10:40 | |
| *** amotoki has quit IRC | 10:41 | |
| *** tosky has quit IRC | 10:41 | |
| *** tosky has joined #openstack-infra | 10:43 | |
| *** eliqiao has quit IRC | 10:47 | |
| *** Jaison has quit IRC | 10:49 | |
| *** tosky has quit IRC | 10:49 | |
| *** eliqiao has joined #openstack-infra | 10:51 | |
| *** javeriak has joined #openstack-infra | 10:51 | |
| *** gongysh_ has quit IRC | 10:52 | |
| *** links has joined #openstack-infra | 10:53 | |
| *** yanyanhu has quit IRC | 10:53 | |
| *** tosky has joined #openstack-infra | 10:54 | |
| *** bhavik has quit IRC | 10:55 | |
| *** sdague has joined #openstack-infra | 10:55 | |
| *** yamamoto has joined #openstack-infra | 10:55 | |
| *** Ravikiran_K has quit IRC | 10:56 | |
| rcarrillocruz | Kiall, mugsie : https://review.openstack.org/#/c/350206/ needs some love :/ | 10:58 |
| zubchick | Hi folks, could you please take a look at the patch https://review.openstack.org/#/c/344311/. It had +1 workflow but still have not merged | 11:09 |
| mugsie | rcarrillocruz: damn pep8 -_- | 11:10 |
| AJaeger | zubchick: is that change dependent on others? | 11:11 |
| *** yamamoto has quit IRC | 11:11 | |
| openstackgerrit | James Slagle proposed openstack-infra/tripleo-ci: Reinstall python-requests in bootstrap script https://review.openstack.org/350521 | 11:11 |
| AJaeger | zubchick: Click on the depends-On and check that all changes it shows are merged. Your job will not merge before those are in. | 11:11 |
| *** javeriak has quit IRC | 11:12 | |
| zubchick | AJaeger: oh, thanks | 11:12 |
| *** jpr has quit IRC | 11:14 | |
| *** jpr has joined #openstack-infra | 11:15 | |
| *** thorst has joined #openstack-infra | 11:16 | |
| openstackgerrit | James Slagle proposed openstack-infra/tripleo-ci: Don't assume an eth1 https://review.openstack.org/350524 | 11:20 |
| tlbr | mordred, pabelanger , hi, please review https://review.openstack.org/#/c/346095/ https://review.openstack.org/#/c/350056/ | 11:20 |
| *** lucasagomes is now known as lucas-hungry | 11:21 | |
| *** rfolco has joined #openstack-infra | 11:22 | |
| *** thorst has quit IRC | 11:25 | |
| *** rhallisey has joined #openstack-infra | 11:25 | |
| *** rfolco has quit IRC | 11:26 | |
| *** yolanda has quit IRC | 11:27 | |
| *** javeriak has joined #openstack-infra | 11:29 | |
| openstackgerrit | Csaba Henk proposed openstack-infra/project-config: Add jobs for Manila glusterfs, glusterfs-native drivers with Heketi layout https://review.openstack.org/287632 | 11:31 |
| openstackgerrit | Csaba Henk proposed openstack-infra/project-config: templatize Manila GlusterFS jobs https://review.openstack.org/350120 | 11:31 |
| *** yolanda has joined #openstack-infra | 11:31 | |
| openstackgerrit | James Slagle proposed openstack-infra/tripleo-ci: Don't assume an eth1 https://review.openstack.org/350524 | 11:34 |
| *** thorst has joined #openstack-infra | 11:41 | |
| *** jkilpatr has joined #openstack-infra | 11:41 | |
| *** amitgandhinz has joined #openstack-infra | 11:44 | |
| *** sshnaidm is now known as sshnaidm|afk | 11:44 | |
| *** coolsvap_ is now known as coolsvap | 11:45 | |
| *** dprince has joined #openstack-infra | 11:45 | |
| *** amitgandhinz has quit IRC | 11:46 | |
| *** tpsilva has joined #openstack-infra | 11:48 | |
| *** markvoelker has joined #openstack-infra | 11:52 | |
| *** yamamoto has joined #openstack-infra | 11:54 | |
| *** psilvad has joined #openstack-infra | 11:55 | |
| *** yamamoto has quit IRC | 11:59 | |
| *** rossella_s has quit IRC | 12:00 | |
| *** yamamoto has joined #openstack-infra | 12:00 | |
| *** rossella_s has joined #openstack-infra | 12:00 | |
| *** psilvad has quit IRC | 12:01 | |
| openstackgerrit | James Slagle proposed openstack-infra/tripleo-ci: Make $TRIPLEO_ROOT overrideable https://review.openstack.org/350539 | 12:03 |
| *** psilvad has joined #openstack-infra | 12:04 | |
| *** yamamoto has quit IRC | 12:05 | |
| *** weshay has joined #openstack-infra | 12:07 | |
| *** javeriak has quit IRC | 12:08 | |
| *** acoles_ is now known as acoles | 12:09 | |
| *** psilvad has quit IRC | 12:10 | |
| *** yamamoto has joined #openstack-infra | 12:11 | |
| *** raildo has joined #openstack-infra | 12:11 | |
| *** ccamacho is now known as ccamacho|lunch | 12:11 | |
| *** sigmavirus_away is now known as sigmavirus | 12:14 | |
| *** signed8bit has joined #openstack-infra | 12:16 | |
| *** xyang1 has joined #openstack-infra | 12:16 | |
| *** narayrak has joined #openstack-infra | 12:17 | |
| *** chem has quit IRC | 12:18 | |
| *** ramishra has quit IRC | 12:18 | |
| *** chem has joined #openstack-infra | 12:18 | |
| *** edmondsw has joined #openstack-infra | 12:19 | |
| *** witek has joined #openstack-infra | 12:22 | |
| *** binbincong has quit IRC | 12:22 | |
| witek | hello | 12:23 |
| witek | I would like to disable all services and enable only 4 in jenkins job definition | 12:24 |
| witek | how should I make it? | 12:24 |
| witek | https://review.openstack.org/#/c/350454 | 12:24 |
| *** gordc has joined #openstack-infra | 12:25 | |
| AJaeger | sdague: can you help witek? I think this needs OVERRIDE_ENABLED_SERVICES for devstack-gate ^ | 12:25 |
| *** rfolco has joined #openstack-infra | 12:25 | |
| *** tqtran has joined #openstack-infra | 12:25 | |
| *** baoli has joined #openstack-infra | 12:26 | |
| vponomaryov | witek: example: https://github.com/openstack-infra/project-config/blob/eef784ea/jenkins/jobs/manila.yaml#L91 | 12:26 |
| witek | great, thanks | 12:26 |
| *** baoli_ has joined #openstack-infra | 12:27 | |
| openstackgerrit | Witold Bedyk proposed openstack-infra/project-config: Disable services for tempest gate https://review.openstack.org/350454 | 12:29 |
| *** tqtran has quit IRC | 12:30 | |
| *** gouthamr has joined #openstack-infra | 12:30 | |
| *** baoli has quit IRC | 12:30 | |
| *** pt_15 has joined #openstack-infra | 12:31 | |
| AJaeger | witek, LGTM | 12:32 |
| *** zhurong has joined #openstack-infra | 12:33 | |
| witek | thanks Andreas | 12:34 |
| *** sshnaidm|afk is now known as sshnaidm | 12:34 | |
| HenryG | Where/how can I find periodic job results? | 12:35 |
| *** daemontool_ has joined #openstack-infra | 12:35 | |
| AJaeger | healthcheck in general - or logs.openstack.org/periodic | 12:37 |
| *** sdake has joined #openstack-infra | 12:37 | |
| *** daemontool has quit IRC | 12:37 | |
| AJaeger | HenryG: One EXample (needs job to use proper reporting): http://status.openstack.org/openstack-health/#/g/build_queue/periodic?groupKey=build_queue&searchJob=translation | 12:38 |
| HenryG | thanks AJaeger | 12:38 |
| AJaeger | Or directly http://logs.openstack.org/periodic/zaqar-ui-propose-translation-update/ | 12:38 |
| *** narayrak has quit IRC | 12:41 | |
| *** rlandy has joined #openstack-infra | 12:41 | |
| HenryG | AJaeger: I see this: http://grafana.openstack.org/dashboard/db/neutron-failure-rate?panelId=4&fullscreen | 12:43 |
| *** lucas-hungry is now known as lucasagomes | 12:43 | |
| *** bhunter71 has quit IRC | 12:43 | |
| HenryG | ^^ seems to indicate periodic-neutron-py34-with-neutron-lib-master has a problem | 12:43 |
| HenryG | But: http://status.openstack.org/openstack-health/#/job/periodic-neutron-py34-with-neutron-lib-master | 12:43 |
| *** javeriak has joined #openstack-infra | 12:45 | |
| *** javeriak has quit IRC | 12:46 | |
| *** piet_ has joined #openstack-infra | 12:46 | |
| *** amitgandhinz has joined #openstack-infra | 12:47 | |
| *** julim has quit IRC | 12:48 | |
| *** sarob has joined #openstack-infra | 12:48 | |
| *** yamamoto has quit IRC | 12:49 | |
| openstackgerrit | Ivan Udovichenko proposed openstack-infra/project-config: Add new projects https://review.openstack.org/347047 | 12:51 |
| *** matt-borland has joined #openstack-infra | 12:51 | |
| *** annegentle has joined #openstack-infra | 12:52 | |
| *** narayrak has joined #openstack-infra | 12:52 | |
| *** amitgandhinz has quit IRC | 12:52 | |
| *** sarob has quit IRC | 12:53 | |
| *** tosky has quit IRC | 12:53 | |
| *** yaume has quit IRC | 12:54 | |
| *** narayrak has quit IRC | 12:54 | |
| *** narayrak has joined #openstack-infra | 12:54 | |
| *** tosky has joined #openstack-infra | 12:55 | |
| *** jsavak has joined #openstack-infra | 12:56 | |
| *** signed8bit is now known as signed8bit_Zzz | 12:56 | |
| AJaeger | HenryG: health has no data since 1st of August - best to ask mtreinish about that. | 12:56 |
| HenryG | AJaeger: Ah, did not notice the dates, thanks | 12:57 |
| openstackgerrit | Scott DAngelo proposed openstack-infra/project-config: Add experimental Cinder job for multibackend https://review.openstack.org/330678 | 12:57 |
| openstackgerrit | Monty Taylor proposed openstack-infra/shade: Bump heatclient dependency to 1.0.0 https://review.openstack.org/350568 | 12:58 |
| openstackgerrit | Monty Taylor proposed openstack-infra/shade: Add ability to upload duplicate images https://review.openstack.org/349710 | 12:58 |
| openstackgerrit | Monty Taylor proposed openstack-infra/shade: Add a 'meta' passthrough parameter for glance images https://review.openstack.org/350114 | 12:58 |
| *** rhallisey has quit IRC | 13:00 | |
| openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci: Implement undercloud upgrade job - Mitaka -> Newton https://review.openstack.org/346995 | 13:00 |
| *** xarses_ has joined #openstack-infra | 13:00 | |
| *** annegentle has quit IRC | 13:00 | |
| *** burgerk has joined #openstack-infra | 13:03 | |
| *** xarses has quit IRC | 13:03 | |
| *** signed8bit_Zzz is now known as signed8bit | 13:04 | |
| *** piet_ has quit IRC | 13:05 | |
| *** rhallisey has joined #openstack-infra | 13:05 | |
| *** kgiusti has joined #openstack-infra | 13:06 | |
| *** mriedem has joined #openstack-infra | 13:06 | |
| *** ramishra has joined #openstack-infra | 13:11 | |
| *** jcoufal has joined #openstack-infra | 13:12 | |
| *** mdrabe has joined #openstack-infra | 13:12 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/shade: Batch calls to list_floating_ips https://review.openstack.org/315697 | 13:13 |
| *** yamamoto has joined #openstack-infra | 13:13 | |
| *** AJaeger has quit IRC | 13:15 | |
| *** john-davidge has left #openstack-infra | 13:15 | |
| *** zhurong has quit IRC | 13:15 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/shade: Depend on python-heatclient>=1.0.0 https://review.openstack.org/350360 | 13:16 |
| *** zhurong has joined #openstack-infra | 13:16 | |
| *** ccamacho|lunch is now known as ccamacho | 13:17 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/shade: Batch calls to list_floating_ips https://review.openstack.org/315697 | 13:17 |
| openstackgerrit | Monty Taylor proposed openstack-infra/shade: Go ahead and admit that we return Munch objects https://review.openstack.org/282908 | 13:17 |
| openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci: Enable net-iso for ovb jobs https://review.openstack.org/343017 | 13:17 |
| *** burgerk has quit IRC | 13:18 | |
| *** AJaeger has joined #openstack-infra | 13:18 | |
| *** ihrachys has joined #openstack-infra | 13:18 | |
| *** AJaeger has joined #openstack-infra | 13:19 | |
| *** markvoelker has quit IRC | 13:19 | |
| *** esberglu has joined #openstack-infra | 13:19 | |
| openstackgerrit | Emilien Macchi proposed openstack-infra/project-config: Implement tripleo-ci-centos-7-nonha-multinode-upgrades job https://review.openstack.org/350580 | 13:21 |
| *** akscram has quit IRC | 13:23 | |
| *** dkranz has joined #openstack-infra | 13:24 | |
| *** pgadiya has quit IRC | 13:25 | |
| *** akscram has joined #openstack-infra | 13:25 | |
| openstackgerrit | Gabriele Cerami proposed openstack/diskimage-builder: secure_path in sudoers: deal with possible quotes https://review.openstack.org/350582 | 13:25 |
| *** zhurong has quit IRC | 13:25 | |
| *** yamamoto has quit IRC | 13:26 | |
| *** AJaeger has quit IRC | 13:26 | |
| *** zhurong has joined #openstack-infra | 13:26 | |
| *** AJaeger has joined #openstack-infra | 13:27 | |
| *** ihrachys has quit IRC | 13:29 | |
| *** tonytan4ever has joined #openstack-infra | 13:32 | |
| *** mmandel has joined #openstack-infra | 13:32 | |
| *** ttx has quit IRC | 13:32 | |
| *** Ravikiran_K has joined #openstack-infra | 13:33 | |
| *** ttx has joined #openstack-infra | 13:33 | |
| *** ttx has quit IRC | 13:33 | |
| *** ttx has joined #openstack-infra | 13:34 | |
| *** ttx has quit IRC | 13:35 | |
| *** ttx has joined #openstack-infra | 13:36 | |
| *** asselin has joined #openstack-infra | 13:36 | |
| *** _ari_ has joined #openstack-infra | 13:40 | |
| *** thiagop has joined #openstack-infra | 13:40 | |
| *** asettle has quit IRC | 13:41 | |
| *** EricGonczer_ has joined #openstack-infra | 13:41 | |
| openstackgerrit | Merged openstack-infra/puppet-openstackci: Use project_config to get nodepool.yaml path https://review.openstack.org/276964 | 13:44 |
| openstackgerrit | Rob Cresswell proposed openstack-infra/project-config: Add nv test job for Django 1.10 in Django OS Auth https://review.openstack.org/350113 | 13:45 |
| openstackgerrit | Rob Cresswell proposed openstack-infra/project-config: Add nv test job for Django 1.10 in Django OS Auth https://review.openstack.org/350113 | 13:46 |
| *** ldnunes has joined #openstack-infra | 13:46 | |
| *** markvoelker has joined #openstack-infra | 13:46 | |
| openstackgerrit | Rob Cresswell proposed openstack-infra/project-config: Add nv test job for Django 1.10 in Django OS Auth https://review.openstack.org/350113 | 13:47 |
| openstackgerrit | Jeremy Stanley proposed openstack-infra/bindep: Build universal wheels https://review.openstack.org/350592 | 13:47 |
| robcresswell | AJaeger: I updated https://review.openstack.org/350113 to use the xenial node; is my entry correct? I'm a little muddled by the config, whether I Should be using the -nv suffix, or defining voting:false etc. | 13:48 |
| *** amitgandhinz has joined #openstack-infra | 13:48 | |
| *** bhavik has joined #openstack-infra | 13:48 | |
| *** zhurong has quit IRC | 13:48 | |
| *** zhurong has joined #openstack-infra | 13:49 | |
| openstackgerrit | Jeremy Stanley proposed openstack-infra/project-config: Make bindep Python 3.5 unit test jobs vote https://review.openstack.org/350594 | 13:50 |
| *** zz_dimtruck is now known as dimtruck | 13:50 | |
| mtreinish | AJaeger: on which pages, it looks like there is data in the DB from today (in general, maybe not for a particular query though) | 13:51 |
| AJaeger | robcresswell: I commented, the job is now X-ubuntu-xenial, so you need t orename in zuul/layout as well | 13:51 |
| mtreinish | rcarrillocruz: yeah it's a bit weird, I filed it upstream: https://github.com/eclipse/mosquitto/issues/234 | 13:52 |
| rcarrillocruz | ++ | 13:52 |
| *** amitgandhinz has quit IRC | 13:52 | |
| AJaeger | mtreinish: see discussion with HenryG - http://status.openstack.org/openstack-health/#/job/periodic-neutron-py34-with-neutron-lib-master and http://grafana.openstack.org/dashboard/db/neutron-failure-rate?panelId=4&fullscreen | 13:53 |
| mtreinish | rcarrillocruz: I'm not actually sure it's the bind failing though, it only happens if the user mosquitto drops down to internally doesn't have permissions to create the log file | 13:53 |
| openstackgerrit | Rob Cresswell proposed openstack-infra/project-config: Add nv test job for Django 1.10 in Django OS Auth https://review.openstack.org/350113 | 13:53 |
| mtreinish | rcarrillocruz: the error reporting is all broken by not being able to write the log file, so that's definitely the first issue to fix | 13:53 |
| *** eharney has joined #openstack-infra | 13:54 | |
| rcarrillocruz | i even forced the conf file to have 'user root', despite the docs in the file saying it defaults to root it was worth checking | 13:54 |
| rcarrillocruz | since it couldn't bind the port | 13:54 |
| AJaeger | robcresswell: LGTM now | 13:54 |
| rcarrillocruz | but didn't help either | 13:54 |
| openstackgerrit | Merged openstack-infra/puppet-openstackci: Include nodepool::mysql from openstackci https://review.openstack.org/303306 | 13:54 |
| mtreinish | rcarrillocruz: when I added 'user root' to the conf file that worked for me | 13:54 |
| rcarrillocruz | oh, and also set the cap to bind ports on the mosquitto binary | 13:54 |
| robcresswell | AJaeger: Thanks. Will look to move more to Xenial post Newton release. | 13:54 |
| rcarrillocruz | and same issue, so not an apparmor/selinux problem | 13:54 |
| *** akshai has joined #openstack-infra | 13:54 | |
| rcarrillocruz | really? 'root' *and* the log file modified? | 13:55 |
| rcarrillocruz | cos i just tweaked the user 'root' on the conf file and didn't help | 13:55 |
| AJaeger | robcresswell: we're in the process of moving *everything* on master to Xenial... | 13:55 |
| *** shamail has joined #openstack-infra | 13:55 | |
| AJaeger | robcresswell: http://lists.openstack.org/pipermail/openstack-dev/2016-July/099818.html | 13:55 |
| mtreinish | AJaeger: so that looks like it;s caused by the data source limitation with subunit2sql. That job is failing prior to anything that generates subunit, so there is nothing to use for populating the DB | 13:55 |
| mtreinish | AJaeger: the graphite/grafana data comes straight from zuul so it always knows about failed jobs | 13:56 |
| AJaeger | HenryG: ^ | 13:56 |
| AJaeger | mtreinish: thanks for checking! | 13:56 |
| HenryG | mtreinish: AJaeger: ack, thanks | 13:57 |
| mtreinish | AJaeger: it's been a bit thing to fix, but the problem is there isn't a queriable data source straight from zuul that openstack-health can look at and correlate to the other data sources | 13:57 |
| mtreinish | s/bit/big/ | 13:57 |
| *** burgerk has joined #openstack-infra | 13:57 | |
| *** zhurong has quit IRC | 13:57 | |
| robcresswell | AJaeger: Yep, understood, wasn't sure how much of that work was being done by infra vs the project teams moving their own tests (as in that previous patch) | 13:57 |
| mtreinish | jhesketh has been working on a sql reporter for a while to provide that | 13:57 |
| *** rockyg has joined #openstack-infra | 13:57 | |
| HenryG | mtreinish: NP, I found a failed job on logs.o.o to look at | 13:57 |
| *** asettle has joined #openstack-infra | 13:58 | |
| mtreinish | rcarrillocruz: hmm, I'm pretty sure. Let me test it right now | 13:58 |
| *** zhurong has joined #openstack-infra | 13:58 | |
| *** rbrndt has joined #openstack-infra | 13:58 | |
| AJaeger | robcresswell: help is welcome on getting this done! | 13:58 |
| rcarrillocruz | mtreinish: however is, is the thing at firehose.openstack.org working now? need me anything to check | 13:58 |
| rcarrillocruz | ? | 13:58 |
| mtreinish | rcarrillocruz: I think we need to restart germqtt on firehose.o.o | 13:59 |
| mtreinish | rcarrillocruz: there were some issues with the puppet-germqtt module but I think the fix made it through the gate overnight | 13:59 |
| mtreinish | oh, nope it's working now | 13:59 |
| mtreinish | at least on the mqtt port | 13:59 |
| *** krot_sickleave is now known as krotscheck | 14:00 | |
| mtreinish | rcarrillocruz: I'm still having trouble using the websocket port. BUt that could just be an issue with the js client I've been using for testing | 14:00 |
| *** akshai has quit IRC | 14:00 | |
| openstackgerrit | Merged openstack-infra/tripleo-ci: Don't assume an eth1 https://review.openstack.org/350524 | 14:00 |
| *** xarses_ has quit IRC | 14:01 | |
| ttx | fungi: when you have a chance, please set owner for the new "*-release-branch" groups to "Release Managers" so that we can set members from there | 14:01 |
| *** tongli has joined #openstack-infra | 14:01 | |
| mtreinish | rcarrillocruz: you can test it too with: 'mosquitto_sub --topic '#' -h firehose.openstack.org' | 14:01 |
| *** esker has joined #openstack-infra | 14:01 | |
| fungi | ttx: working on it now--will let you know when it's done | 14:01 |
| ttx | fungi: thx! | 14:01 |
| *** akshai has joined #openstack-infra | 14:02 | |
| *** asettle has quit IRC | 14:02 | |
| *** eharney_ has joined #openstack-infra | 14:04 | |
| mtreinish | rcarrillocruz: oh cool, the websocket port looks like it's working too. (it was just a client issue on my end) | 14:04 |
| *** eharney has quit IRC | 14:04 | |
| mtreinish | rcarrillocruz: I think this means all the work items from the spec are complete :) | 14:04 |
| *** tongli has quit IRC | 14:04 | |
| *** tongli has joined #openstack-infra | 14:05 | |
| mtreinish | I still want to get tls ports implemented before we close it out though | 14:05 |
| rcarrillocruz | \o/ | 14:05 |
| mtreinish | we probably don't need it for external random subscribers, but for any internal communication between servers we should force it | 14:06 |
| openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci: Implement undercloud upgrade job - Mitaka -> Newton https://review.openstack.org/346995 | 14:06 |
| mtreinish | fungi, rcarrillocruz: if you want to play with the websocket side I used: http://mitsuruog.github.io/what-mqtt/ to test it | 14:07 |
| rcarrillocruz | mtreinish: firehose doesn't have 'user root' on the conf file | 14:07 |
| rcarrillocruz | it has it commented it out | 14:07 |
| rcarrillocruz | just as it was when i checked it yesterday | 14:07 |
| rcarrillocruz | so i guess what fungi changed about the log file dest fixed somehow the bind port issue | 14:07 |
| *** bhunter71 has joined #openstack-infra | 14:07 | |
| fungi | ttx: all set. just double-checked and all 20 look like they're updated now | 14:08 |
| mtreinish | rcarrillocruz: yeah we set it back to the default location of /var/log/mosquitto/mosquitto.log which the mosquitto user has write permission on | 14:08 |
| mtreinish | that unblocked things | 14:08 |
| fungi | rcarrillocruz: right, for strange reasons it couldn't bind to privileged ports if the log file wasn't writable by the mosquitto user | 14:09 |
| fungi | i'm guessing there's still some sort of subtle bug with their privilege separation code | 14:09 |
| fungi | rcarrillocruz: also the hiera entry for the gerrit public key was all sorts of wrong. it looked like it was formatted as an authorized_keys entry instead of a known_hosts entry | 14:10 |
| fungi | rcarrillocruz: but on subsequent discussion we're all over the map with how we've implemented configuration management of known_hosts entries si i'm going to try to clean that up across all our modules and make it consistent and centralized | 14:11 |
| fungi | gerrit public host key i mean | 14:11 |
| *** bhavik has quit IRC | 14:11 | |
| *** yaume has joined #openstack-infra | 14:11 | |
| *** ayoung has joined #openstack-infra | 14:12 | |
| AJaeger | fungi, corvus, could you put reviewing the constraints/zuul-cloner change on your dance card for today, please? https://review.openstack.org/#/c/349627/ . Other project-config cores are welcome to review as well! | 14:13 |
| openstackgerrit | Jeremy Stanley proposed openstack-infra/project-config: Sign all tarballs and wheels https://review.openstack.org/350603 | 14:13 |
| rcarrillocruz | fungi: i think i copy pasted from zuul hiera yaml file to the firehose yaml straight | 14:14 |
| fungi | rcarrillocruz: probably, though the zuul servers puppet the contents of that file differently than this module | 14:15 |
| fungi | rcarrillocruz: our zuul management concatenates the key material with hostname/address data to build the line rather than just using it as the sole content | 14:16 |
| *** ramishra has quit IRC | 14:16 | |
| rcarrillocruz | right | 14:16 |
| *** EricGonczer_ has quit IRC | 14:16 | |
| fungi | so for now i've added the hostname and addresses to that hiera key which got it working | 14:16 |
| *** EricGonczer_ has joined #openstack-infra | 14:17 | |
| fungi | but ultimately i want to make this consistent everywhere, and also not in private hiera | 14:17 |
| *** oanson has quit IRC | 14:17 | |
| fungi | it can just be one global value in our public hiera and all our servers that need it get it from the same entry that way | 14:17 |
| rcarrillocruz | yeah, cos having a key name with different formatting is bad | 14:17 |
| rcarrillocruz | not sure if we could standardize something about formatting on our key naming | 14:18 |
| *** amotoki has joined #openstack-infra | 14:18 | |
| rcarrillocruz | otherwise we'll cargo cult and copy paste around | 14:18 |
| rcarrillocruz | or maybe document somewhere how should the keys be formatted, be in the puppet manifest docs or in some docs at infra namespace | 14:19 |
| rcarrillocruz | brb, dog walk | 14:19 |
| mtreinish | rcarrillocruz: well that's what I did for puppet-germqtt I just copy and pasted it from puppet-elastic_recheck | 14:20 |
| *** siva_krish has joined #openstack-infra | 14:21 | |
| *** links has quit IRC | 14:23 | |
| *** claudiub has quit IRC | 14:24 | |
| *** dkehn has quit IRC | 14:24 | |
| *** siva_krish has quit IRC | 14:24 | |
| *** jaosorior has quit IRC | 14:25 | |
| *** dkehn_ has quit IRC | 14:25 | |
| pabelanger | fungi: are we ready for 350603? I can +3 now | 14:26 |
| *** siva_krish has joined #openstack-infra | 14:26 | |
| *** yamamoto has joined #openstack-infra | 14:26 | |
| AJaeger | pabelanger: will take 3h+ to merge... | 14:26 |
| pabelanger | AJaeger: okay, I'll wait for checks to finish | 14:28 |
| rcarrillocruz | yep | 14:28 |
| rcarrillocruz | i couldn't know if the key was ok or not as the service was not even up :/ | 14:28 |
| *** ramishra has joined #openstack-infra | 14:28 | |
| AJaeger | pabelanger: you can approve now if you like - if it fails, it will not merge ;) | 14:28 |
| rcarrillocruz | anyway, worth specing out the formatting of this stuff at some point | 14:28 |
| *** yamahata has joined #openstack-infra | 14:29 | |
| AJaeger | pabelanger: just wanted to point out that we're not able to see this merge *now* since Zuul has a huge backlog | 14:29 |
| *** EricGonczer_ has quit IRC | 14:31 | |
| fungi | pabelanger: AJaeger: yeah, should be fine though i'll also give the release team a heads up | 14:31 |
| *** ihrachys has joined #openstack-infra | 14:31 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/project-config: Switch tripleo jobs back to rh1 https://review.openstack.org/348879 | 14:31 |
| *** dkehn has joined #openstack-infra | 14:31 | |
| *** EricGonczer_ has joined #openstack-infra | 14:31 | |
| pabelanger | AJaeger: Nope, that works for me. Lets wait until zuul votes then line up a time to land | 14:31 |
| *** yamamoto has quit IRC | 14:33 | |
| *** siva_krish has quit IRC | 14:33 | |
| openstackgerrit | Doug Hellmann proposed openstack-infra/project-config: comment out skip for manual release https://review.openstack.org/350616 | 14:33 |
| *** jlanoux has quit IRC | 14:37 | |
| *** rbuzatu has quit IRC | 14:37 | |
| *** xarses_ has joined #openstack-infra | 14:38 | |
| *** dkehn_ has joined #openstack-infra | 14:38 | |
| *** burgerk has quit IRC | 14:38 | |
| fungi | pabelanger: AJaeger: i gave the release team a heads up, so feel free to approve whenever | 14:38 |
| *** caowei has joined #openstack-infra | 14:39 | |
| *** mtanino has joined #openstack-infra | 14:40 | |
| *** amotoki has quit IRC | 14:40 | |
| openstackgerrit | Merged openstack/os-client-config: Add support for deprecating cloud profiles https://review.openstack.org/339988 | 14:42 |
| pabelanger | fungi: AJaeger +3 | 14:42 |
| openstackgerrit | Merged openstack/os-client-config: Add support for listing a cloud as shut down https://review.openstack.org/343112 | 14:43 |
| *** burgerk has joined #openstack-infra | 14:43 | |
| openstackgerrit | Merged openstack/os-client-config: Refactor fix magic in get_one_cloud() https://review.openstack.org/331418 | 14:44 |
| *** ramishra has quit IRC | 14:44 | |
| *** Swami has joined #openstack-infra | 14:45 | |
| openstackgerrit | Monty Taylor proposed openstack/os-client-config: Add the new DreamCompute cloud https://review.openstack.org/339989 | 14:45 |
| *** amotoki has joined #openstack-infra | 14:45 | |
| *** toabctl has quit IRC | 14:46 | |
| *** ramishra has joined #openstack-infra | 14:46 | |
| pabelanger | AJaeger: we should land 347941 to avoid endless rebasing too | 14:46 |
| pabelanger | xenial-docs jobs | 14:46 |
| *** toabctl has joined #openstack-infra | 14:47 | |
| *** annegentle has joined #openstack-infra | 14:47 | |
| *** hieulq_ has joined #openstack-infra | 14:47 | |
| openstackgerrit | Doug Hellmann proposed openstack-infra/project-config: provide a manual override mode https://review.openstack.org/350616 | 14:48 |
| *** amitgandhinz has joined #openstack-infra | 14:49 | |
| pabelanger | clarkb: did you hear back from ovh about nodepool capacity? | 14:49 |
| dhellmann | hmm, openstackgerrit doesn't seem to be posting in #openstack-release | 14:49 |
| openstackgerrit | David Shrewsbury proposed openstack-infra/nodepool: Add config support for ZooKeeper servers https://review.openstack.org/350623 | 14:49 |
| AJaeger | pabelanger: I wanted to wait for clarkb - but he's on vacation now AFAIK, so let's go forward with it... | 14:49 |
| pabelanger | AJaeger: Ah, I missed the PTO memo :) | 14:50 |
| AJaeger | pabelanger: there's also https://review.openstack.org/#/c/348777/ - which might need a zuul restart for recalculation... | 14:50 |
| *** ramishra has quit IRC | 14:50 | |
| *** eharney_ is now known as eharney | 14:51 | |
| *** berendt has joined #openstack-infra | 14:52 | |
| *** mdrabe has quit IRC | 14:53 | |
| *** ramishra has joined #openstack-infra | 14:53 | |
| pabelanger | AJaeger: okay, lets land them and see how calculation looks | 14:53 |
| *** rbuzatu has joined #openstack-infra | 14:53 | |
| pabelanger | we restarted zuul ~12 hours ago FYI | 14:53 |
| *** amitgandhinz has quit IRC | 14:53 | |
| pabelanger | AJaeger: I also have https://review.openstack.org/#/c/349119/ which will provide a more realistic reflection of the nodes we are using in graphite / grafana. Today we are missing sub-nodes in the calculations | 14:55 |
| *** karthikp_ has joined #openstack-infra | 14:55 | |
| openstackgerrit | Merged openstack-infra/project-config: update tag-release git command to fall back to master https://review.openstack.org/350214 | 14:55 |
| *** igorn has joined #openstack-infra | 14:56 | |
| *** siva_krish has joined #openstack-infra | 14:58 | |
| *** asettle has joined #openstack-infra | 14:59 | |
| fungi | dhellmann: odd. it's present in the channel, and it's saying things in here | 14:59 |
| fungi | what did you push that it should have echoed in there? | 14:59 |
| *** pradk_ has joined #openstack-infra | 14:59 | |
| openstackgerrit | Merged openstack/os-client-config: Fix precedence for pass-in options https://review.openstack.org/350346 | 15:00 |
| *** esker has quit IRC | 15:00 | |
| *** mdrabe has joined #openstack-infra | 15:01 | |
| *** nmagnezi has quit IRC | 15:02 | |
| *** asettle has quit IRC | 15:03 | |
| *** hieulq__ has joined #openstack-infra | 15:05 | |
| *** jlanoux has joined #openstack-infra | 15:06 | |
| *** amotoki has quit IRC | 15:06 | |
| *** hieulq_ has quit IRC | 15:07 | |
| *** roxanaghe has joined #openstack-infra | 15:07 | |
| *** PollyZ has joined #openstack-infra | 15:07 | |
| *** akshai has quit IRC | 15:09 | |
| openstackgerrit | Sagi Shnaidman proposed openstack-infra/tripleo-ci: WIP: DONT MERGE TESTING https://review.openstack.org/316436 | 15:09 |
| *** dprince has quit IRC | 15:10 | |
| *** siva_krish has quit IRC | 15:12 | |
| openstackgerrit | Jeremy Stanley proposed openstack-infra/zuul: Cloner: Update heads and tags from the cache https://review.openstack.org/350637 | 15:14 |
| fungi | jeblair: ^ that just crossed my mind when talking to bhunter71 about local git caches on our workers | 15:14 |
| phschwartz | Is there a way to get the id of the nodepool node a job is running on? I want to display it in a notification for build failure. | 15:14 |
| fungi | phschwartz: yep, it's a parameter and it's displayed at the start of the job | 15:14 |
| fungi | phschwartz: assuming you mean the nodepool id and not the nova uuid | 15:15 |
| phschwartz | fungi: yes, the nodepool id | 15:15 |
| *** dimtruck is now known as zz_dimtruck | 15:15 | |
| *** zz_dimtruck is now known as dimtruck | 15:16 | |
| *** julim has joined #openstack-infra | 15:16 | |
| phschwartz | fungi: I see it displayed in the jenkins log, just can't find a parameter in the param list that matches it | 15:16 |
| fungi | phschwartz: as an example, we attach the build_node as metadata on elasticsearch documents | 15:16 |
| openstackgerrit | Ben Nemec proposed openstack-infra/tripleo-ci: Retry stack-delete in case of failure https://review.openstack.org/350640 | 15:16 |
| fungi | phschwartz: so that lines of output can be correlated by build_node | 15:16 |
| *** dkehn has quit IRC | 15:17 | |
| phschwartz | fungi: awesome, let me look at the upstream job and I can go from there | 15:17 |
| *** akshai has joined #openstack-infra | 15:18 | |
| rcarrillocruz | Shrews , mordred , mugsie : https://review.openstack.org/#/c/350206/ revert landed | 15:18 |
| rcarrillocruz | i'l recheck shade changes | 15:18 |
| rcarrillocruz | yolanda: ^ | 15:18 |
| yolanda | ah great | 15:19 |
| *** priteau has quit IRC | 15:19 | |
| *** dkehn_ has quit IRC | 15:22 | |
| *** amotoki has joined #openstack-infra | 15:22 | |
| fungi | phschwartz: skimming http://git.openstack.org/cgit/openstack-infra/log_processor/tree/log_processor/client.py i think it's emitted in zeromq events as the "node_name" parameter | 15:24 |
| fungi | phschwartz: that may not be reflected in any zuul/jenkins job parameters though | 15:25 |
| *** dimtruck is now known as zz_dimtruck | 15:25 | |
| *** caowei has quit IRC | 15:26 | |
| *** dkehn has joined #openstack-infra | 15:26 | |
| *** ihrachys has quit IRC | 15:29 | |
| *** ihrachys has joined #openstack-infra | 15:30 | |
| *** bhavik has joined #openstack-infra | 15:31 | |
| *** vvalderrv has joined #openstack-infra | 15:31 | |
| *** dkehn_ has joined #openstack-infra | 15:32 | |
| *** zhurong has quit IRC | 15:33 | |
| *** tlbr has quit IRC | 15:36 | |
| *** piet has joined #openstack-infra | 15:39 | |
| *** zz_dimtruck is now known as dimtruck | 15:40 | |
| openstackgerrit | Matthew Treinish proposed openstack-infra/gerritbot: Add gerrit stream over MQTT support to gerritbot https://review.openstack.org/350651 | 15:40 |
| openstackgerrit | Matthew Treinish proposed openstack-infra/gerritbot: WIP: Add foreground mode https://review.openstack.org/350652 | 15:40 |
| openstackgerrit | Artur Zarzycki proposed openstack-infra/project-config: Add fuel-ccp-ingress repository https://review.openstack.org/350653 | 15:40 |
| mtreinish | fungi, rcarrillocruz: did something happen to germqtt? it looks like the stream stopped | 15:41 |
| *** mikelk has quit IRC | 15:42 | |
| *** vincentll has quit IRC | 15:42 | |
| fungi | mtreinish: the process is still running, last started 06:43 utc today | 15:43 |
| *** jpeeler has quit IRC | 15:43 | |
| fungi | mtreinish: and gerrit claims there's an open connection for the account | 15:43 |
| fungi | open since 06:43:14 utc | 15:43 |
| mtreinish | fungi: oops, nm it was on my end | 15:44 |
| rcarrillocruz | haven't touched it | 15:44 |
| rcarrillocruz | ah k | 15:44 |
| mtreinish | I was running with a very narrow topic, and forgot it was that terminal | 15:44 |
| fungi | mtreinish: cool, was gonna say i also see traffic back and forth for port 29418 | 15:44 |
| mtreinish | fungi, rcarrillocruz: if you're curious I've got a gerritbot running with ^^^ over mqtt up on #throwaway-gerritbot-mqtt-test-channel | 15:47 |
| *** armax has joined #openstack-infra | 15:47 | |
| mtreinish | it seems to be working well | 15:47 |
| rcarrillocruz | sweet | 15:48 |
| openstackgerrit | Matthew Treinish proposed openstack-infra/gerritbot: Add gerrit stream over MQTT support to gerritbot https://review.openstack.org/350651 | 15:48 |
| openstackgerrit | Matthew Treinish proposed openstack-infra/gerritbot: WIP: Add foreground mode https://review.openstack.org/350652 | 15:48 |
| fungi | mtreinish: harlowja might be interested in that for the errbot-based rewrite too | 15:48 |
| mtreinish | hmm, maybe not so well, it just missed it's own commits ^^^ :( | 15:49 |
| fungi | though we should almost certainly treat firehose.o.o as a proof of concept for a while still, and not actually make any production services depend on it | 15:49 |
| mtreinish | heh, and now it's stack tracing | 15:49 |
| fungi | like all good software | 15:49 |
| mtreinish | fungi: sure, but I figure it'd be good to have something on the client side to show off | 15:49 |
| fungi | mtreinish: absolutely | 15:50 |
| *** amitgandhinz has joined #openstack-infra | 15:50 | |
| *** ihrachys has quit IRC | 15:51 | |
| *** mdrabe_ has joined #openstack-infra | 15:51 | |
| *** shashank_hegde has joined #openstack-infra | 15:52 | |
| *** PollyZ has quit IRC | 15:53 | |
| *** Sukhdev has joined #openstack-infra | 15:54 | |
| *** mdrabe has quit IRC | 15:55 | |
| *** amitgandhinz has quit IRC | 15:55 | |
| *** PollyZ has joined #openstack-infra | 15:58 | |
| *** akshai has quit IRC | 15:59 | |
| *** ramishra has quit IRC | 15:59 | |
| *** rossella_s has quit IRC | 16:00 | |
| openstackgerrit | Jeremy Stanley proposed openstack-infra/puppet-jenkins: Optionally manage .ssh/known_hosts on slaves https://review.openstack.org/350663 | 16:00 |
| *** rossella_s has joined #openstack-infra | 16:00 | |
| *** annegentle has quit IRC | 16:01 | |
| *** rbrndt has quit IRC | 16:01 | |
| *** annegentle has joined #openstack-infra | 16:02 | |
| *** ddieterly has joined #openstack-infra | 16:04 | |
| *** itisha has joined #openstack-infra | 16:04 | |
| zaro | morning | 16:04 |
| zaro | fungi: are we waiting for another gerrit restart to approve the its-storyboard plugin? | 16:06 |
| *** rbrndt has joined #openstack-infra | 16:06 | |
| *** jistr is now known as jistr|biab | 16:06 | |
| zaro | i'm wondering how we can puppet it onto review-dev.o.o before that happens? | 16:07 |
| openstackgerrit | Matthew Treinish proposed openstack-infra/gerritbot: Add gerrit stream over MQTT support to gerritbot https://review.openstack.org/350651 | 16:07 |
| openstackgerrit | Matthew Treinish proposed openstack-infra/gerritbot: WIP: Add foreground mode https://review.openstack.org/350652 | 16:07 |
| mtreinish | ^^^ I think that does it now | 16:07 |
| mtreinish | it seems to be working very reliably in the throwaway channel | 16:07 |
| * mtreinish shouldn't jinx himself | 16:07 | |
| *** psachin has quit IRC | 16:11 | |
| openstackgerrit | Dean Troyer proposed openstack/os-client-config: Add release notes for 1.19.0 release https://review.openstack.org/350672 | 16:11 |
| fungi | zaro: i don't think we're waiting to approve it, we just need to plan to do a lengthy reindex outage the next time we have a planned gerrit maintenance | 16:11 |
| *** shashank_hegde has quit IRC | 16:12 | |
| fungi | zaro: i'll go ahead and review those outstanding changes now | 16:12 |
| *** adrian_otto has joined #openstack-infra | 16:13 | |
| *** dtantsur is now known as dtantsur|afk | 16:15 | |
| zaro | fungi: i would suggest _not_ doing a reindex just for change https://review.openstack.org/344519 | 16:16 |
| zaro | that's not required to get storyboard gerrit integration work | 16:16 |
| zaro | i just set it to the same topic because it's related, but not necessary | 16:17 |
| zaro | the only requirement for gerrit storyboard integration is a gerrit restart after those changes are approved | 16:17 |
| zaro | sorry if that was not clear. | 16:17 |
| *** Goneri has quit IRC | 16:18 | |
| fungi | yeah, i figured it was just the one adding story/task ids to the index that would need a reindex run | 16:18 |
| *** PollyZ has quit IRC | 16:18 | |
| zaro | i think we should approve them now so puppet can put it all on review-dev.o.o. that would allow testing before we do for realz | 16:19 |
| *** roxanaghe has quit IRC | 16:20 | |
| *** hieulq__ has quit IRC | 16:21 | |
| *** jistr|biab is now known as jistr | 16:22 | |
| jeblair | notmorgan, jhesketh, mordred: sorry, i accidentally self-approved a change without review: https://review.openstack.org/350165 | 16:25 |
| jeblair | (it was on the tip of a series that needed re-approval after a rebase) | 16:25 |
| notmorgan | jeblair: probably not an issue. | 16:25 |
| *** ccamacho is now known as ccamacho|out | 16:26 | |
| zaro | yay! force gerrit online reindex finally got approved https://gerrit-review.googlesource.com/77890 | 16:26 |
| jeblair | zaro: that is wonderful news :) | 16:27 |
| *** tphummel has joined #openstack-infra | 16:27 | |
| mordred | \o/ | 16:27 |
| *** tqtran has joined #openstack-infra | 16:27 | |
| Zara | :D | 16:27 |
| *** dansmith has quit IRC | 16:27 | |
| *** Apoorva has joined #openstack-infra | 16:27 | |
| notmorgan | wooooot | 16:27 |
| notmorgan | jeblair: trying to read that patch before coffee has settled... | 16:28 |
| *** dansmith has joined #openstack-infra | 16:28 | |
| *** dansmith is now known as Guest20988 | 16:28 | |
| notmorgan | jeblair: notmorgan.waitUntilCoffeeSettled() | 16:28 |
| *** yaume has quit IRC | 16:28 | |
| jeblair | notmorgan: haha :) | 16:28 |
| notmorgan | jeblair: at a glance, it looks ok, fwiw | 16:29 |
| *** Guest20988 is now known as dansmith | 16:29 | |
| *** akshai has joined #openstack-infra | 16:30 | |
| jeblair | i think the most objectionable part of it is adding a parameter to zuul.launcher.server.AnsibleLaunchServer.runAnsible() which the method does not actually need, but the test subclass does. | 16:30 |
| jpmaxman | just fyi fungi I looked the extensions that are actually registered and I get a much smaller number: http://paste.openstack.org/show/547807/ | 16:30 |
| jeblair | i really hate that, but it makes the testing very simple and reliable. i at least left a comment about it. :) | 16:30 |
| notmorgan | jeblair: yay comments | 16:31 |
| jeblair | notmorgan: what test were you working on re-enabling? i was thinking of working on testScheduler.test_parallel_changes... | 16:31 |
| fungi | jpmaxman: that looks much closer to what the version page reports | 16:31 |
| *** tqtran has quit IRC | 16:31 | |
| notmorgan | jeblair: jeblair the tests in test_connection | 16:31 |
| jpmaxman | some of those are built in | 16:32 |
| jpmaxman | not sure they get reported | 16:32 |
| fungi | jpmaxman: i wonder why some of those don't appear on the version page (or whether they're builtins( | 16:32 |
| *** akshai_ has joined #openstack-infra | 16:32 | |
| fungi | yeah, what i was thinking too | 16:32 |
| jeblair | notmorgan: cool, so no collision there | 16:32 |
| notmorgan | jeblair: basically it was/is a small set of tests, but requires a full config, figured it was a good place to work | 16:32 |
| jeblair | ya | 16:32 |
| fungi | jpmaxman: had any of that list turned up as upgrade problems so far? | 16:32 |
| notmorgan | jeblair: one of them enables with no changes, and the other two are just chasing config changes down. | 16:32 |
| jpmaxman | I'm running some tests | 16:33 |
| *** jsavak has quit IRC | 16:33 | |
| jpmaxman | right now the nocaptcha shows | 16:33 |
| fungi | ooh! | 16:33 |
| *** dprince has joined #openstack-infra | 16:33 | |
| jpmaxman | but even if you succesfully verify | 16:33 |
| jpmaxman | it says you haven't | 16:33 |
| *** narayrak has quit IRC | 16:33 | |
| jpmaxman | so trying to figure out where the conflict is | 16:33 |
| jpmaxman | generally it is looking good | 16:33 |
| *** jsavak has joined #openstack-infra | 16:33 | |
| openstackgerrit | Merged openstack-infra/puppet-gerrit: Add configurations for gerrit its plugins https://review.openstack.org/330922 | 16:33 |
| notmorgan | jeblair: but i wanted to be sure it was fixture-per-class or per-test-file or whatevert before building all the config things. | 16:33 |
| *** ddieterly is now known as ddieterly[away] | 16:34 | |
| pabelanger | fungi: rcarrillocruz: did I hear you say infracloud was back online? | 16:34 |
| *** sdague has quit IRC | 16:34 | |
| pabelanger | mordred: notmorgan: do you mind adding 349119 to your list of nodepool reviews? | 16:35 |
| *** akshai has quit IRC | 16:35 | |
| *** roxanaghe has joined #openstack-infra | 16:35 | |
| fungi | pabelanger: rcarrillocruz said he received some promising e-mails but hadn't had time to follow up on them due to unanticipated plague | 16:36 |
| *** Hal has quit IRC | 16:36 | |
| fungi | i don't recall if there was more recent news since the weekend | 16:36 |
| *** rcernin has quit IRC | 16:38 | |
| notmorgan | pabelanger: i don't have an issue with it. lgtm (but I can't +2 it) | 16:38 |
| fungi | zaro: all the topic:storyboard-its changes are approved now except your wip one enabling the plugin in production | 16:38 |
| pabelanger | thanks, so it sounds like it might be online for the mid-cycle | 16:38 |
| notmorgan | pabelanger: +1'd. | 16:38 |
| pabelanger | Still working on getting travel approval for QAInfraNewtonSprint and wanting to confirm the tasks we'll be working on. Otherwise, I'll work on them remotely | 16:39 |
| mordred | pabelanger: +A | 16:39 |
| pabelanger | notmorgan: mordred: great thanks | 16:39 |
| openstackgerrit | Merged openstack-infra/system-config: Configure its-storyboard plugin https://review.openstack.org/330925 | 16:39 |
| *** ddieterly[away] is now known as ddieterly | 16:40 | |
| *** Sukhdev has quit IRC | 16:40 | |
| *** mhickey has quit IRC | 16:41 | |
| openstackgerrit | Merged openstack-infra/project-config: networking-midonet: Add periodic jobs for stable branches https://review.openstack.org/334830 | 16:42 |
| mtreinish | AJaeger: heh, the throwaway channel is pretty boring, it's just my local bot running with: https://review.openstack.org/350651 using firehose.o.o | 16:42 |
| *** ramishra has joined #openstack-infra | 16:43 | |
| *** Teresita-Warrior has joined #openstack-infra | 16:43 | |
| *** sdague has joined #openstack-infra | 16:43 | |
| *** e0ne has quit IRC | 16:44 | |
| openstackgerrit | Merged openstack-infra/system-config: Add storyboard task to Gerrit tracking id https://review.openstack.org/344519 | 16:45 |
| *** roxanaghe has quit IRC | 16:45 | |
| Zara | =D | 16:45 |
| *** lucasagomes is now known as lucas-dinner | 16:45 | |
| Zara | fungi: thanks for reviewing the gerrit-storyboard integration patches! | 16:46 |
| fungi | sorry for not getting to them earlier! | 16:46 |
| jpmaxman | fungi any idea what this one did? https://www.mediawiki.org/wiki/Extension:Mantle | 16:46 |
| pabelanger | mordred: jeblair: Is there any plans / schedule for zuulv3 in place for the mid-cycle? | 16:46 |
| *** roxanaghe has joined #openstack-infra | 16:47 | |
| *** arosen12 has joined #openstack-infra | 16:47 | |
| fungi | jpmaxman: looking at https://www.mediawiki.org/w/index.php?title=Extension:Mantle&oldid=1958470 i can't imagine what we'd have used it for, if anything ever | 16:47 |
| fungi | "Currently the main purpose of this extension is to implement compilation into JavaScript of templates sent the server to the client by ResourceLoader, while the HTML templating language RFC is implemented." | 16:47 |
| jpmaxman | ok well I'm going to kill it and see if anything or anyone complains :D | 16:48 |
| fungi | jpmaxman: looks like at one time it was used by the mobilefrontend extension, according to that | 16:48 |
| *** javeriak has joined #openstack-infra | 16:48 | |
| fungi | jpmaxman: so presumably a dependency since dropped in newer versions of the extension | 16:48 |
| *** furlongm_ has quit IRC | 16:49 | |
| *** furlongm_ has joined #openstack-infra | 16:50 | |
| *** hieulq_ has joined #openstack-infra | 16:51 | |
| *** arosen12 has left #openstack-infra | 16:51 | |
| *** amitgandhinz has joined #openstack-infra | 16:52 | |
| jpmaxman | ok - well i'd already disabled mobile front end as it was complaining | 16:52 |
| jpmaxman | may need to get the newest version of that | 16:52 |
| *** krotscheck is now known as kro_focused | 16:53 | |
| *** adrian_otto has quit IRC | 16:53 | |
| jeblair | pabelanger: i don't think so yet. we should regroup and see where we are. | 16:55 |
| openstackgerrit | Michael Krotscheck proposed openstack-infra/project-config: Created DSVM Job for NPM Projects https://review.openstack.org/348056 | 16:55 |
| *** Hal has joined #openstack-infra | 16:55 | |
| *** roxanaghe has quit IRC | 16:56 | |
| *** amitgandhinz has quit IRC | 16:56 | |
| *** adrian_otto has joined #openstack-infra | 16:56 | |
| *** jpr has quit IRC | 16:57 | |
| *** roxanaghe has joined #openstack-infra | 16:57 | |
| pabelanger | jeblair: perfect | 16:57 |
| *** roxanaghe has quit IRC | 16:57 | |
| *** roxanaghe has joined #openstack-infra | 16:57 | |
| *** asettle has joined #openstack-infra | 17:00 | |
| *** pcaruana has quit IRC | 17:00 | |
| *** tqtran has joined #openstack-infra | 17:00 | |
| dhellmann | fungi : oh, you know, I think I was looking for the link to https://review.openstack.org/350616 to show up there but I forgot it's in the project-config repo now. Is there a way to have the bot look at paths on files, so we could have it announce changes to those scripts in #openstack-release too? | 17:00 |
| *** tqtran_ has joined #openstack-infra | 17:01 | |
| *** annegentle has quit IRC | 17:02 | |
| fungi | dhellmann: i don't think so, but would need to dig in the bot source to find out if it has a filter option like that | 17:02 |
| *** degorenko_ has joined #openstack-infra | 17:02 | |
| *** tesseract- has quit IRC | 17:03 | |
| *** tqtran has quit IRC | 17:04 | |
| zaro | fungi: thanks! | 17:05 |
| *** rhedlind has joined #openstack-infra | 17:06 | |
| *** sputnik13 has joined #openstack-infra | 17:06 | |
| *** openstackgerrit_ has joined #openstack-infra | 17:06 | |
| *** nmagnezi has joined #openstack-infra | 17:07 | |
| *** degorenko_ has quit IRC | 17:08 | |
| *** openstackgerrit_ has quit IRC | 17:08 | |
| *** piet has quit IRC | 17:09 | |
| *** jlanoux has quit IRC | 17:09 | |
| mgagne | with grafyaml, is it supposed to support configuration of multiple datasources? | 17:10 |
| fungi | pabelanger: ^ ? | 17:10 |
| mgagne | looks like internal structure supports multiple sources but syntax in config file makes it impossible to configure more | 17:10 |
| fungi | might just be that it was designed with that in mind but we've not tried it so implementation there is actually incomplete? | 17:10 |
| *** akshai_ has quit IRC | 17:11 | |
| mgagne | oh =( | 17:11 |
| *** karthikp_ has quit IRC | 17:11 | |
| mgagne | or it's one datasource per file? | 17:12 |
| *** coreyob has quit IRC | 17:12 | |
| *** rbuzatu has quit IRC | 17:12 | |
| mgagne | I thought it would be datasources: instead of datasource: :O | 17:12 |
| openstackgerrit | Merged openstack-infra/shade: Depend on python-heatclient>=1.0.0 https://review.openstack.org/350360 | 17:13 |
| *** sarob has joined #openstack-infra | 17:13 | |
| fungi | mgagne: looks like we only use one anyway http://git.openstack.org/cgit/openstack-infra/project-config/tree/grafana/datasource.yaml | 17:13 |
| *** jpr1 has joined #openstack-infra | 17:13 | |
| *** coreyob has joined #openstack-infra | 17:14 | |
| mgagne | yea, but looking at how dashboard are managed, it looks like it's one dashboard/datasource per file | 17:14 |
| fungi | so basically we're just using it to graph data sourced from our graphite server | 17:14 |
| mgagne | right, I happen to have more than one =) | 17:14 |
| fungi | we don't currently have any other sources of data backing grafana so it was probably just not a design consideration (but doesn't seem like something we would reject as an enhancement) | 17:15 |
| *** yamahata has quit IRC | 17:15 | |
| mgagne | will test before complaining more =) | 17:16 |
| *** piet has joined #openstack-infra | 17:16 | |
| pabelanger | mgagne: fungi: I cannot remember. Let me check | 17:17 |
| pabelanger | mgagne: fungi: however, it would be easy to add a unit test to validate it | 17:17 |
| *** PollyZ has joined #openstack-infra | 17:17 | |
| mgagne | ok, multiple files = multiple datasources | 17:18 |
| mgagne | confirmed on my side, sorry for the noise :D | 17:18 |
| *** jsavak has quit IRC | 17:18 | |
| pabelanger | ya, that sounds right | 17:18 |
| *** jsavak has joined #openstack-infra | 17:18 | |
| kro_focused | Not to be all whiny and stuff, but how much would the gate improve if we had dib create a devstack image instead of installing it with devstack-gate? | 17:19 |
| *** eharney has quit IRC | 17:20 | |
| fungi | kro_focused: poorly since then we wouldn't be able to test devstack-gate | 17:20 |
| mgagne | :D | 17:20 |
| mordred | I think it depends, honestly | 17:20 |
| mordred | the things that make up devstack, nova and friends, would not really be able to use that | 17:21 |
| mordred | but things that use devstack clouds as a way to test themselves would likely benefit | 17:21 |
| fungi | depends on if kro_focused means preinstall devstack or just prerun the bits of devstack-gate that don't run stack.sh | 17:21 |
| mordred | since they largely only need to consume the configured endpoint of a cloud | 17:21 |
| mordred | fungi: yah. I mean, I could see using a pre-installed devstack in shade's functional tests, for instance | 17:22 |
| *** dimtruck is now known as zz_dimtruck | 17:22 | |
| mordred | since shade does not care at all about the installation of opentsack | 17:22 |
| kro_focused | Well, I'm looking at a 4 hour wait on most jobs right now, and DSVM nodes seem to take the longest. | 17:22 |
| mordred | but that's also a fringe player in terms of gate usage | 17:22 |
| jeblair | kro_focused, mordred, fungi: yeah, i think it might be worth a shot. | 17:22 |
| jeblair | kro_focused: oh, well, i don't think it's going to radically change *that* | 17:22 |
| *** Swami has quit IRC | 17:22 | |
| mordred | jeblair: have a "yesterday's devstack" image type perhaps? | 17:22 |
| jeblair | mordred: right | 17:22 |
| jeblair | kro_focused: this would perhaps reduce a 10 minute wait in the job to probably a minute or two? | 17:23 |
| fungi | mordred: the way to really improve that is probably what we were discussing a while back, something like nodepool that sets up throwaway tenants^Wprojects in an existing openstack deployment and then assigns them to jobs where a worker can make api calls | 17:23 |
| fungi | kro_focused: we don't have dsvm nodes | 17:23 |
| *** Swami has joined #openstack-infra | 17:23 | |
| *** csomerville has quit IRC | 17:23 | |
| kro_focused | fungi: Potato, potato. | 17:23 |
| fungi | kro_focused: the dsvm jobs are just using plain old trusty and xenial (and sometimes centos or fedora) nodes used to run other non-dsvm-based jobs | 17:24 |
| jeblair | kro_focused: it would be helpful within jobs that use it, but i don't think it would make a significant impact on the overall system utilization | 17:24 |
| *** slaweq__ has quit IRC | 17:24 | |
| fungi | kro_focused: so it's more accurate to say that there are a bunch of dsvm jobs waiting for node assignments, but there are a bunch of jobs in general waiting for node assignments | 17:24 |
| *** cody-somerville has joined #openstack-infra | 17:24 | |
| *** cody-somerville has joined #openstack-infra | 17:24 | |
| *** ddieterly is now known as ddieterly[away] | 17:24 | |
| fungi | kro_focused: more likely the jobs you see completing sooner are using ubuntu-trusty and most of the dsvm jobs for master are running on ubuntu-xenial and we might not be building enough of the latter | 17:25 |
| kro_focused | Ya know, this seems like a topic for the cross-project technical goals thread. | 17:25 |
| openstackgerrit | Merged openstack-infra/system-config: Install its-storyboard plugin on review-dev.o.o https://review.openstack.org/340605 | 17:25 |
| dhellmann | fungi : ok, it's not worth worrying about right now. If we're rewriting those bots we can add it then | 17:25 |
| *** slaweq_ has joined #openstack-infra | 17:25 | |
| *** hieulq_ has quit IRC | 17:25 | |
| kro_focused | fungi: I don't have enough insight into pool queue usage. | 17:25 |
| openstackgerrit | John L. Villalovos proposed openstack-infra/project-config: Ironic: Make ironic-multitenant-network a voting job https://review.openstack.org/350703 | 17:26 |
| *** hieulq_ has joined #openstack-infra | 17:27 | |
| *** e0ne has joined #openstack-infra | 17:27 | |
| fungi | kro_focused: yeah, i think one thing we're lacking right now is grafana graphs of job nodes by node type | 17:27 |
| openstackgerrit | Merged openstack-infra/shade: Go ahead and admit that we return Munch objects https://review.openstack.org/282908 | 17:27 |
| AJaeger | mtreinish: it was far too noisy for me ;) | 17:28 |
| *** e0ne has quit IRC | 17:28 | |
| pabelanger | fungi: kro_focused: People see value in knowing how many ubuntu-trusty nodes are running / building / deleting? | 17:29 |
| mordred | woot! shade patch landed | 17:29 |
| pabelanger | fungi: kro_focused: If so, I was thinking about updating http://grafana.openstack.org/dashboard/db/nodepool to include that info | 17:29 |
| kro_focused | pabelanger: Well, as a measure of "Here's the number of consumers vs. the provided number of resources" | 17:30 |
| fungi | kro_focused: yeah, looks like as a raw statistic we've got ~3x as many trusty nodes as xenial at the moment http://paste.openstack.org/show/547814/ | 17:30 |
| kro_focused | pabelanger: Call it "Zuulconomics". More demand for xenial makes the cost of jobs higher ;) | 17:30 |
| kro_focused | Hah. And one lonely precise-node | 17:31 |
| pabelanger | right, but once we migrate more jobs away from ubuntu-trusty, xenial will become dominate | 17:31 |
| fungi | kro_focused: nodepool is supposed to be interpreting the work requests in zuul's gearman to determine how many of what node type should be built | 17:31 |
| mtreinish | harlowja: I'm running that gerritbot patch off of firehose.o.o right now on #throwaway-gerritbot-mqtt-test-channel if you're curious | 17:31 |
| harlowja | mtreinish nice | 17:31 |
| pabelanger | kro_focused: fungi: https://review.openstack.org/#/q/status:open+project:openstack-infra/project-config+branch:master+topic:newton-xenial should help with that | 17:32 |
| *** slaweq_ has quit IRC | 17:32 | |
| kro_focused | Nice | 17:32 |
| *** shashank_hegde has joined #openstack-infra | 17:33 | |
| jeblair | fungi: nodepool does do that, however, it has been altered to slightly favor less-used nodes | 17:34 |
| fungi | yep, or else under heavy demand we can end up never building them | 17:34 |
| *** javeriak has quit IRC | 17:35 | |
| fungi | if they represent a smaller ratio than one out of our usable aggregate quota | 17:35 |
| *** rockyg has quit IRC | 17:36 | |
| pabelanger | clarkb has an email out to ovh about upping our resource in ovh-gra1, I should follow up on that | 17:36 |
| jeblair | our current deficit is 3x as many xenial as trusty | 17:37 |
| jeblair | (and yes, that appears to be the inverse of what fungi said our actual is. i don't know if that's coincidence or the way the allocator math works out) | 17:37 |
| *** daemontool_ has quit IRC | 17:38 | |
| *** slaweq__ has joined #openstack-infra | 17:38 | |
| jeblair | anyway, zuulv3 will be fifo. | 17:38 |
| *** slaweq__ has quit IRC | 17:38 | |
| bkero | It might be worth pointing out that afaict the dsvm jobs haven't been switched over yet. https://etherpad.openstack.org/p/xenial-newton-tasks | 17:38 |
| openstackgerrit | Michael Krotscheck proposed openstack-infra/project-config: Added documentation draft jobs for nodejs-based projects https://review.openstack.org/346130 | 17:39 |
| openstackgerrit | Michael Krotscheck proposed openstack-infra/project-config: Added documentation publish jobs for the JavaScript SDK https://review.openstack.org/346131 | 17:39 |
| openstackgerrit | Jeremy Stanley proposed openstack-infra/system-config: Maintain Gerrit host keys in a global known_hosts https://review.openstack.org/350709 | 17:40 |
| *** jsavak has quit IRC | 17:40 | |
| fungi | jeblair: mtreinish: rcarrillocruz: ^ as discussed last night, spitball system-config patch first to model the plan before i go implementing it across individual modules (so far only the puppet-jenkins depends-on is proposed) | 17:40 |
| *** jsavak has joined #openstack-infra | 17:40 | |
| openstackgerrit | Matthew Treinish proposed openstack-infra/gerritbot: Add gerrit stream over MQTT support to gerritbot https://review.openstack.org/350651 | 17:40 |
| openstackgerrit | Matthew Treinish proposed openstack-infra/gerritbot: WIP: Add foreground mode https://review.openstack.org/350652 | 17:40 |
| mtreinish | fungi: cool, so that will just pass the content and the module is expected to write that to the known_hosts file? | 17:42 |
| *** PollyZ has quit IRC | 17:43 | |
| *** e0ne has joined #openstack-infra | 17:43 | |
| fungi | mtreinish: that's the idea anyway. it gets rid of a lot of the hackery of composing individual known_hosts entries for starters. and having additional known_hosts entries on a system doesn't hurt even if they're unused | 17:43 |
| openstackgerrit | Merged openstack-infra/project-config: Move {name}-tarball jobs over to xenial https://review.openstack.org/348777 | 17:44 |
| *** shardy has quit IRC | 17:44 | |
| fungi | mtreinish: we could in theory add common entries for other things to the same publig global hiera entry in the future | 17:44 |
| fungi | which is why i didn't give it a gerrit-specific name | 17:44 |
| mtreinish | fungi: ah, yeah makes sense | 17:45 |
| *** piet has quit IRC | 17:45 | |
| fungi | mtreinish: there were a few ways i considered going about it, but ultimately this seemed like the least complex and easiest to understand/update | 17:47 |
| kro_focused | Is this a thing we want? https://review.openstack.org/#/c/349098/ | 17:47 |
| *** mat128 is now known as mat128|afk | 17:47 | |
| * kro_focused is starting to think no, because canonical locations of clouds.yaml | 17:47 | |
| fungi | kro_focused: is it likely to be something other than /etc/clouds.yaml? | 17:48 |
| kro_focused | fungi: You tell me ;) | 17:48 |
| fungi | er, /etc/openstack/clouds.yaml i mean | 17:48 |
| * kro_focused wants his jobs not to break if infra decides to change that location. | 17:48 | |
| mordred | I don't expect it to ever be anywhere else | 17:48 |
| fungi | this would be less infra deciding and more os-client-config deciding anyway | 17:49 |
| fungi | that's its default expected location afaik | 17:49 |
| mordred | although I do not have a problem witih exporting a variable with the canonical location in it | 17:49 |
| fungi | i doubt it hurts to add it, though it will likely always remain entirely redundant, and it does slightly add to environment pollution | 17:50 |
| openstackgerrit | John L. Villalovos proposed openstack-infra/project-config: Ironic: Make ironic-multitenant-network a voting job https://review.openstack.org/350703 | 17:51 |
| *** amitgandhinz has joined #openstack-infra | 17:53 | |
| *** Na3iL has quit IRC | 17:53 | |
| *** julim has quit IRC | 17:53 | |
| *** yamahata has joined #openstack-infra | 17:54 | |
| *** nmagnezi has quit IRC | 17:54 | |
| *** claudiub has joined #openstack-infra | 17:54 | |
| *** EricGonc_ has joined #openstack-infra | 17:55 | |
| *** Na3iL has joined #openstack-infra | 17:56 | |
| *** julim has joined #openstack-infra | 17:56 | |
| openstackgerrit | Michael Krotscheck proposed openstack-infra/project-config: Splitting current/deprecated integ. test https://review.openstack.org/349077 | 17:57 |
| *** EricGonczer_ has quit IRC | 17:57 | |
| *** amitgandhinz has quit IRC | 17:57 | |
| *** harlowja has quit IRC | 17:58 | |
| openstackgerrit | Merged openstack-infra/project-config: Sign all tarballs and wheels https://review.openstack.org/350603 | 17:59 |
| *** ddieterly[away] is now known as ddieterly | 18:00 | |
| *** e0ne has quit IRC | 18:00 | |
| *** harlowja has joined #openstack-infra | 18:00 | |
| *** hieulq_ has quit IRC | 18:02 | |
| *** sambetts is now known as sambetts|afk | 18:02 | |
| *** fernnest has joined #openstack-infra | 18:04 | |
| *** piet has joined #openstack-infra | 18:04 | |
| *** rockyg has joined #openstack-infra | 18:05 | |
| *** PollyZ has joined #openstack-infra | 18:06 | |
| *** degorenko is now known as _degorenko|afk | 18:07 | |
| *** asettle has quit IRC | 18:07 | |
| *** tonytan4ever has quit IRC | 18:08 | |
| *** zz_dimtruck is now known as dimtruck | 18:09 | |
| *** tosky has quit IRC | 18:09 | |
| *** rbrndt has quit IRC | 18:10 | |
| *** eeiden_ has joined #openstack-infra | 18:10 | |
| *** cody-somerville has quit IRC | 18:10 | |
| *** cody-somerville has joined #openstack-infra | 18:11 | |
| *** acoles is now known as acoles_ | 18:11 | |
| *** skipp has quit IRC | 18:12 | |
| *** stewie925 has quit IRC | 18:12 | |
| *** rbuzatu has joined #openstack-infra | 18:13 | |
| *** eeiden has quit IRC | 18:13 | |
| openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci: WIP - Implement undercloud upgrade job - Mitaka -> Newton https://review.openstack.org/346995 | 18:14 |
| *** rbrndt has joined #openstack-infra | 18:15 | |
| *** pvaneck has joined #openstack-infra | 18:16 | |
| *** ddieterly is now known as ddieterly[away] | 18:16 | |
| *** ddieterly[away] is now known as ddieterly | 18:17 | |
| *** rbuzatu has quit IRC | 18:17 | |
| *** stewie925 has joined #openstack-infra | 18:17 | |
| *** rbrndt has quit IRC | 18:18 | |
| *** yamahata has quit IRC | 18:18 | |
| *** yamahata has joined #openstack-infra | 18:19 | |
| *** adrian_otto has quit IRC | 18:21 | |
| *** claudiub has quit IRC | 18:21 | |
| *** jpich has quit IRC | 18:22 | |
| *** electrofelix has quit IRC | 18:22 | |
| *** HeOS has quit IRC | 18:26 | |
| *** Na3iL has quit IRC | 18:26 | |
| *** rwsu has quit IRC | 18:27 | |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul: Correct default value for hold-following-changes https://review.openstack.org/350725 | 18:28 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul: Tests: Restore support for examining commits in fake builds https://review.openstack.org/350726 | 18:28 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul: Restore test_parallel_changes https://review.openstack.org/350727 | 18:28 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul: Remove unused Inheritable class https://review.openstack.org/350728 | 18:28 |
| bkero | prometheanfire: are these going to be added to a review? https://bpaste.net/show/39020cb4839c | 18:29 |
| * bkero is doing tab culling and have a diff open | 18:29 | |
| rcarrillocruz | fungi: ++, will check in a bit | 18:29 |
| *** jaosorior has joined #openstack-infra | 18:29 | |
| *** jaosorior has quit IRC | 18:30 | |
| *** skipp has joined #openstack-infra | 18:32 | |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul: Tests: Improve support in fake builds https://review.openstack.org/350726 | 18:33 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul: Restore test_parallel_changes https://review.openstack.org/350727 | 18:33 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul: Remove unused Inheritable class https://review.openstack.org/350728 | 18:33 |
| *** diltram has joined #openstack-infra | 18:36 | |
| diltram | hey, is there any way to debug docs-gate? | 18:36 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul: Tests: remove FakeWorker https://review.openstack.org/350730 | 18:36 |
| diltram | or where I can find info how to setup my own docs gate | 18:37 |
| jeblair | diltram: do you have a link to a failed build? | 18:37 |
| *** amotoki has quit IRC | 18:37 | |
| diltram | jeblair: https://review.openstack.org/#/c/297423/ | 18:38 |
| openstackgerrit | Merged openstack-infra/tripleo-ci: Add opstools candidate repo https://review.openstack.org/325095 | 18:38 |
| diltram | jeblair: exactly here | 18:38 |
| diltram | http://docs-draft.openstack.org/23/297423/8/check/gate-octavia-docs-ubuntu-xenial/497ca74//doc/build/html/devref/flows.html | 18:38 |
| *** ayoung has quit IRC | 18:38 | |
| diltram | we should have an image generated using pydotplus and there is nothing | 18:39 |
| *** annegentle has joined #openstack-infra | 18:39 | |
| diltram | and we don't know what is going one | 18:39 |
| *** ccamacho|out is now known as ccamacho | 18:39 | |
| *** ddieterly is now known as ddieterly[away] | 18:40 | |
| prometheanfire | bkero: no, that's just so everything is happy with puppet failing | 18:42 |
| bkero | ah | 18:42 |
| *** piet has quit IRC | 18:43 | |
| *** rbuzatu has joined #openstack-infra | 18:43 | |
| prometheanfire | that's git stash material, not commit :D | 18:43 |
| bkero | yeah | 18:44 |
| bkero | I was wondering if it was necessary to make gentoo work with system-config, and if it was, if it was going to land | 18:45 |
| prometheanfire | it's not, I need to start work on the broken puppet peices is all | 18:45 |
| bkero | ah | 18:45 |
| bkero | I should be familiar with that if I can get my build to go that far :) | 18:46 |
| prometheanfire | bkero: what's odd is that you can see in the logs that puppet apply is failing, but DIB doesn't care | 18:46 |
| *** julim has quit IRC | 18:46 | |
| bkero | prometheanfire: that is odd | 18:46 |
| prometheanfire | but for me it works out, since that's not what I'm targeting for a fix yet | 18:48 |
| *** rbuzatu has quit IRC | 18:48 | |
| fungi | diltram: the associated job logs for that build can be found by replacing docs-draft with logs like so: http://logs.openstack.org/23/297423/8/check/gate-octavia-docs-ubuntu-xenial/497ca74/ | 18:48 |
| diltram | fungi: great, thank you very much :) | 18:49 |
| fungi | diltram: this looks suspect... http://logs.openstack.org/23/297423/8/check/gate-octavia-docs-ubuntu-xenial/497ca74/console.html#_2016-08-03_07_13_41_329963 | 18:50 |
| *** piet has joined #openstack-infra | 18:50 | |
| fungi | "dot exited with error: Warning: using box for unknown shape cylinder | 18:50 |
| fungi | diltram: ^ | 18:50 |
| diltram | fungi: yep - Database [shape=cylinder]; | 18:51 |
| fungi | and then "*** stack smashing detected ***: dot terminated" (not sure if the warning was related to the stack smashing) | 18:51 |
| rcarrillocruz | Shrews , mordred : shade is back to passing func no tests, revert on designate code fixed it | 18:52 |
| rcarrillocruz | mugsie: ^ thx! | 18:52 |
| mordred | rcarrillocruz: woot! | 18:52 |
| *** _sarob has joined #openstack-infra | 18:53 | |
| *** amitgandhinz has joined #openstack-infra | 18:54 | |
| openstackgerrit | Merged openstack-infra/zuul: Add docstrings to the model https://review.openstack.org/348928 | 18:55 |
| *** jed56 has quit IRC | 18:55 | |
| *** adrian_otto has joined #openstack-infra | 18:56 | |
| openstackgerrit | Merged openstack-infra/zuul: Add an internals doc to the documentation https://review.openstack.org/349035 | 18:56 |
| openstackgerrit | Merged openstack-infra/zuul: Rename BasePipelineManager to PipelineManager https://review.openstack.org/349036 | 18:56 |
| *** sarob has quit IRC | 18:56 | |
| *** claudiub has joined #openstack-infra | 18:56 | |
| *** eharney has joined #openstack-infra | 18:58 | |
| *** aviau has quit IRC | 18:58 | |
| *** aviau has joined #openstack-infra | 18:58 | |
| *** amitgandhinz has quit IRC | 18:58 | |
| *** karthik__ has joined #openstack-infra | 19:01 | |
| *** claudiub has quit IRC | 19:02 | |
| prometheanfire | bkero: let me know if anything else comes up :D | 19:02 |
| *** jsavak has quit IRC | 19:03 | |
| bkero | prometheanfire: Sure thing. I'm still running into the perm denied because of /dev/shm | 19:03 |
| prometheanfire | bkero: your output from earlier didn't seem to include my debug code | 19:04 |
| *** xarses_ is now known as xarses | 19:04 | |
| bkero | prometheanfire: Yeah, that's why I usually put some obvious non-whitespace characters in it, to emphasize it | 19:04 |
| prometheanfire | bkero: there :P https://gist.github.com/prometheanfire/9d9a25b9504bbe0dca0f131c0debdd6a | 19:05 |
| bkero | yeah, running now | 19:06 |
| prometheanfire | k | 19:06 |
| *** fifieldt has quit IRC | 19:07 | |
| prometheanfire | again, gate is working... | 19:07 |
| bkero | That's fantastic | 19:07 |
| prometheanfire | :P | 19:07 |
| *** tonytan4ever has joined #openstack-infra | 19:08 | |
| *** EricGonczer_ has joined #openstack-infra | 19:09 | |
| *** rbrndt has joined #openstack-infra | 19:09 | |
| *** mctaylor has joined #openstack-infra | 19:10 | |
| bkero | prometheanfire: http://paste.openstack.org/show/547833/ | 19:11 |
| *** EricGonc_ has quit IRC | 19:13 | |
| *** tonytan4ever has quit IRC | 19:13 | |
| prometheanfire | bkero: ah I wonder if it's exiting because it returned 1 | 19:14 |
| anteaya | fungi: was taking a personal day today then it occured to me that you or any other infra root could pull together the electroral roll for the requirements ptl election that is scheduled to begin on friday | 19:14 |
| prometheanfire | it didn't do the stat for instance | 19:14 |
| anteaya | fungi: so i thought I would come online with a reminder and to express my gratitude for your help | 19:15 |
| anteaya | so thank you | 19:15 |
| *** Sukhdev has joined #openstack-infra | 19:15 | |
| diltram | fungi: ok, I know that this error doesn't matter, but I found out that this docs gate is not running tox -edocs from project but some "gate" scripts in venv only | 19:15 |
| anteaya | any time you have a chance to create that and email it to myself and dhellmann between now and friday morning is wonderful | 19:15 |
| anteaya | thanks | 19:15 |
| bkero | prometheanfire: http://paste.openstack.org/show/547834/ | 19:16 |
| bkero | I commented out the grep | 19:16 |
| prometheanfire | so no stat output? | 19:16 |
| prometheanfire | that seems odd | 19:17 |
| prometheanfire | maybe ls -l /dev and see what it says shm is? | 19:17 |
| prometheanfire | in the debug | 19:17 |
| *** ddieterly[away] is now known as ddieterly | 19:17 | |
| bkero | prometheanfire: http://paste.openstack.org/show/547835/ | 19:18 |
| *** fifieldt has joined #openstack-infra | 19:19 | |
| prometheanfire | bkero: looks like it's a normal directory | 19:19 |
| bkero | yup | 19:19 |
| prometheanfire | thought you said it was mounted :| | 19:19 |
| *** jsavak has joined #openstack-infra | 19:19 | |
| bkero | it is | 19:19 |
| prometheanfire | then why doesn't /proc/mounts know about it | 19:20 |
| bkero | tmpfs 5.9G 1.3M 5.9G 1% /dev/shm | 19:20 |
| bkero | it does | 19:20 |
| bkero | [bkero@Whitefall shm]$ grep /dev/shm /proc/mounts | 19:20 |
| bkero | tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 | 19:20 |
| prometheanfire | but in the script it returned 1 | 19:20 |
| bkero | looks like it returned 0 to me | 19:20 |
| prometheanfire | line 448 http://paste.openstack.org/show/547833/ | 19:20 |
| bkero | dib overall returned 1, yes | 19:21 |
| prometheanfire | oh | 19:21 |
| prometheanfire | well, why did it exit before | 19:21 |
| prometheanfire | line 440 is the first | 19:21 |
| prometheanfire | there we did the grep | 19:22 |
| prometheanfire | grep returned nothing (caused it to exit 1), causing the script to exit and then dib to exit | 19:22 |
| prometheanfire | all fails | 19:22 |
| *** _ari_ has quit IRC | 19:22 | |
| * bkero runs with -x | 19:22 | |
| *** mctaylor has quit IRC | 19:22 | |
| prometheanfire | k | 19:22 |
| fungi | anteaya: gladly! i'll get that running now | 19:22 |
| bkero | greghaynes: did you say you created a dib IRC channel? | 19:22 |
| greghaynes | bkero: we need to propose the channel creation stuffs | 19:23 |
| greghaynes | we decided were going to do it, no one has | 19:23 |
| bkero | ah | 19:23 |
| *** zul_ has joined #openstack-infra | 19:23 | |
| prometheanfire | greghaynes: I'd join :D | 19:23 |
| * bkero sitting in #diskimage-builder | 19:23 | |
| anteaya | fungi: thank you | 19:23 |
| fungi | diltram: correct, the job basically runs `tox -e venv python setup.py build_pshinx` per the consistent testing interface documented on governance.openstack.org | 19:23 |
| greghaynes | we really should request access before folks join otherwise its a pain to get ops back | 19:24 |
| fungi | diltram: so you should in theory be able to reproduce the problem that way | 19:24 |
| prometheanfire | greghaynes: if it's unregistered that helps | 19:24 |
| anteaya | fungi: I think you already have all the information you need to run that, yes? | 19:24 |
| diltram | fungi: this is a problem :P it's not running our additionall script which is generating that images | 19:24 |
| fungi | anteaya: correct, i'll simply rerun it the same way i did last time | 19:24 |
| jeblair | diltram: sorry, i got pulled away from the keyboard! thanks fungi for jumping in :) | 19:24 |
| anteaya | fungi: okay great, thank you | 19:24 |
| anteaya | fungi: going offline again | 19:24 |
| fungi | diltram: right, you probably need to implement it as a sphinx extension | 19:25 |
| fungi | diltram: rather than additional commands in tox or a wrapper script | 19:25 |
| diltram | ok, thx fungi, jeblair for your help, I'm gonna take a look into that | 19:25 |
| bkero | greghaynes: what's the process for registering a new channel? https://wiki.openstack.org/wiki/IRC just says ask in here or on ML | 19:25 |
| greghaynes | bkero: you now know more than I do about it :) | 19:26 |
| jeblair | bkero: http://docs.openstack.org/infra/system-config/irc.html | 19:26 |
| bkero | jeblair: thank you! | 19:26 |
| jeblair | bkero: all of the services we help run are documented here: http://docs.openstack.org/infra/system-config/ | 19:26 |
| fungi | or should be, and if they're not, that's a bug ;) | 19:27 |
| * bkero nods | 19:27 | |
| *** asettle has joined #openstack-infra | 19:27 | |
| bkero | bah, meeting. I'll do the channel stuff when I get back | 19:28 |
| jeblair | fungi, mtreinish: is there an outstanding patch to add mqtt docs? | 19:28 |
| *** mctaylor has joined #openstack-infra | 19:28 | |
| *** piet has quit IRC | 19:28 | |
| fungi | jeblair: i think the spec is incomplete until they're added, at a minimum | 19:28 |
| jeblair | fungi: yes, the spec calls for them: http://specs.openstack.org/openstack-infra/infra-specs/specs/firehose.html#documentation | 19:29 |
| *** piet has joined #openstack-infra | 19:29 | |
| fungi | now that it's actually running, i agree it's time (even overdue) for service documentation in system-config | 19:29 |
| fungi | i'll whip something up and get mtreinish to correct it | 19:29 |
| jeblair | fungi: i just thought i saw something from mtreinish along the lines of "i think that's all the work items in the spec done" | 19:29 |
| fungi | oh! i'll check outstanding changes first then | 19:30 |
| *** jsavak has quit IRC | 19:30 | |
| jeblair | which, i suppose may technically be true since there isn't a work item for "write docs" but i would expect that to be implied by "deploy server" | 19:31 |
| *** jsavak has joined #openstack-infra | 19:31 | |
| fungi | yeah, not finding any but starting to flesh it out now | 19:31 |
| *** tesseract- has joined #openstack-infra | 19:31 | |
| *** rockyg has quit IRC | 19:33 | |
| *** HeOS has joined #openstack-infra | 19:34 | |
| *** clenimar has quit IRC | 19:34 | |
| *** jcoufal has quit IRC | 19:34 | |
| *** clenimar has joined #openstack-infra | 19:35 | |
| *** zul_ has quit IRC | 19:35 | |
| *** gouthamr has quit IRC | 19:35 | |
| *** zul_ has joined #openstack-infra | 19:35 | |
| *** tesseract- has quit IRC | 19:37 | |
| rhedlind | zuul question: zuul was killed by the system because it ran out of memory. Now whenever I try to restart zuul it won't create the zuul.pid file. No hints in the log files as to why. Is there another location where zuul keeps a persistent state that might make a new zuul process think another copy is already running? | 19:38 |
| jeblair | rhedlind: you may need to delete a stale .pid file | 19:39 |
| jeblair | rhedlind: ours is in /var/run/zuul/zuul.pid | 19:39 |
| rhedlind | jeblair: checked there. nothing in that folder | 19:39 |
| *** eeiden_ has quit IRC | 19:39 | |
| *** rwsu has joined #openstack-infra | 19:40 | |
| jeblair | rhedlind: you might try running as the zuul user with '-d' to tell it not to daemonize and see what happens | 19:40 |
| *** bhavik has quit IRC | 19:41 | |
| openstackgerrit | John L. Villalovos proposed openstack-infra/project-config: Ironic: Make ironic-multitenant-network a voting job https://review.openstack.org/350703 | 19:41 |
| *** ihrachys has joined #openstack-infra | 19:41 | |
| mordred | also, are you accidentally trying to start it as the wrong user? | 19:42 |
| *** asettle has quit IRC | 19:42 | |
| rhedlind | I'm trying to start it as root | 19:43 |
| *** gouthamr has joined #openstack-infra | 19:45 | |
| *** ayoung has joined #openstack-infra | 19:46 | |
| *** rockyg has joined #openstack-infra | 19:46 | |
| *** roxanaghe has quit IRC | 19:48 | |
| *** rbuzatu has joined #openstack-infra | 19:49 | |
| *** tonytan4ever has joined #openstack-infra | 19:49 | |
| openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci: Add undercloud job in cistatus https://review.openstack.org/350763 | 19:50 |
| *** Goneri has joined #openstack-infra | 19:51 | |
| mtreinish | fungi: I forgot to push the doc patch, give me one sec I need to double check it and then I'll push it up | 19:51 |
| fungi | mtreinish: oh, you already have one? i'll pop it back off my queue and review yours in that case | 19:52 |
| mtreinish | fungi: yeah, it's been sitting in my local repo since yesterday, I just never pushed it | 19:53 |
| fungi | excellent | 19:53 |
| mtreinish | fungi: of course I doubt it's complete enough :) But it's a starting point if nothing else | 19:53 |
| *** rbuzatu has quit IRC | 19:53 | |
| *** amitgandhinz has joined #openstack-infra | 19:54 | |
| *** jpr1 has quit IRC | 19:54 | |
| openstackgerrit | Matthew Treinish proposed openstack-infra/system-config: Add docs for firehose.o.o https://review.openstack.org/350764 | 19:55 |
| mtreinish | fungi: ^^^ | 19:55 |
| therve | I just discovered openstack-health thanks to Armando's email | 19:56 |
| mordred | mtreinish: so, can I use that mqtt broker for my home automation hub? | 19:56 |
| therve | How reliable is the data in there? | 19:56 |
| therve | It seems to be incorrect for heat at least | 19:56 |
| *** diltram has left #openstack-infra | 19:56 | |
| *** mhickey has joined #openstack-infra | 19:56 | |
| *** Ravikiran_K has quit IRC | 19:57 | |
| mtreinish | mordred: well you can because you have heira access. But we locked down write access to a single user | 19:57 |
| mordred | mtreinish: ah - ok | 19:57 |
| mordred | mtreinish: I did not catch the user existence from the doc | 19:57 |
| mtreinish | mordred: feel free to -1 on that if you think I should mention it's read only for external users | 19:58 |
| *** dprince has quit IRC | 19:58 | |
| *** amitgandhinz has quit IRC | 19:59 | |
| *** asettle has joined #openstack-infra | 19:59 | |
| *** asettle has quit IRC | 19:59 | |
| mtreinish | therve: the data is fine, there are a few limitations though. It's based on subunit2sql which means it only has data if the run emitted subunit. So if something failed before devstack (or the test runner in non-dsvm jobs) a db record may not have been created for it | 19:59 |
| mordred | mtreinish: nope! I +2'd it already | 19:59 |
| mordred | mtreinish: I think it's great, because I now know more than I did before I read it | 19:59 |
| mordred | all docs can always be improved :) | 19:59 |
| *** rossella_s has quit IRC | 20:00 | |
| therve | mtreinish, Hum, ok. I wonde if we emit subunit properly then | 20:00 |
| therve | Because we didn't have a failure today, which sounds suspicious :) | 20:00 |
| *** rossella_s has joined #openstack-infra | 20:00 | |
| mtreinish | therve: do you have a link to a run which failed? | 20:01 |
| therve | mtreinish, http://logs.openstack.org/29/350029/4/check/gate-heat-dsvm-functional-orig-mysql-lbaasv2/ec0ec83/ | 20:02 |
| *** permalac has quit IRC | 20:02 | |
| pabelanger | fungi: jeblair: if projects wanted to send data to graphite.o.o from their test jobs, is that something we'd entertain? | 20:02 |
| therve | There is almost empty testrepository.subunit.gz file | 20:02 |
| mtreinish | therve: yeah so the subunit output there (in logs/testrepository.subunit) only has devstack | 20:03 |
| * therve nods | 20:03 | |
| fungi | pabelanger: what sort of data would they send? | 20:03 |
| fungi | pabelanger: also, curious how we'd wrangle access controls around that | 20:03 |
| mordred | pabelanger: SpamapS was working on a thing to collect some metrics and then have a something read the results and inject them into graphite iirc | 20:03 |
| mtreinish | therve: whatever you're using to run testr has to do something like: http://git.openstack.org/cgit/openstack-infra/devstack-gate/tree/functions.sh#n657 | 20:03 |
| mordred | fungi: ^^ | 20:03 |
| *** permalac has joined #openstack-infra | 20:03 | |
| mordred | SpamapS: did you ever get anywhere with that? | 20:03 |
| therve | mtreinish, Awesome, thank you | 20:04 |
| mtreinish | therve: to put the subunit output in the right place | 20:04 |
| pabelanger | fungi: for example statsd gauges, or timers | 20:04 |
| mtreinish | mordred: SpamapS's qa bluebrint/spec is still open. He never finished it :( | 20:04 |
| pabelanger | mordred: SpamapS: interesting | 20:04 |
| * pabelanger googles | 20:04 | |
| mtreinish | mordred: http://specs.openstack.org/openstack/qa-specs/specs/devstack/counter-inspection.html | 20:04 |
| mtreinish | pabelanger: ^^^ | 20:05 |
| mordred | yah. that | 20:05 |
| mordred | that's basically looking at the same problem space | 20:05 |
| fungi | pabelanger: statsd gauges and timers of what though? | 20:05 |
| pabelanger | cool, okay so there has been some discussion are that | 20:05 |
| mordred | fungi: well, in the case that SpamapS was working on, it was db usage counters from running tests - so that things like total number of rows or total number of queries could be tracked | 20:05 |
| pabelanger | fungi: in the case of tripleo-ci, they are using it do track how long sub tasks run when a job is running | 20:06 |
| mordred | as a way to spot significant usage increases | 20:06 |
| fungi | pabelanger: could they record timing data in a subunit file? | 20:06 |
| *** matrohon has joined #openstack-infra | 20:07 | |
| fungi | this sounds more like the stuff we're stashing in the subunit2sql db | 20:07 |
| pabelanger | fungi: for example: http://git.openstack.org/cgit/openstack-infra/tripleo-ci/tree/toci_instack.sh#n22 | 20:07 |
| pabelanger | fungi: I think if we proposed a different solution, they would be open to it | 20:07 |
| pabelanger | for now, they have some random graphite server on the public web collecting the data | 20:07 |
| pabelanger | which, I have no idea, what they use it for | 20:07 |
| pabelanger | trying to track that down now | 20:07 |
| *** bhunter71 has quit IRC | 20:08 | |
| *** EricGonc_ has joined #openstack-infra | 20:08 | |
| mtreinish | pabelanger: that does sound a lot like what subunit2sql is tracking for test runs | 20:08 |
| fungi | SpamapS' partial work notwistanding, we don't currently have jobs tracking granular timing data in graphite but we _do_ have them tracking it in subunit2sql (openstack-health, stackviz, et cetera) | 20:08 |
| mordred | mtreinish: ++ | 20:08 |
| pabelanger | perfect | 20:08 |
| pabelanger | then I can start with that | 20:08 |
| pabelanger | ya, stackviz would work well I think | 20:09 |
| fungi | they they wrote their tests as a tempest plugin (no idea how feasible that is) they could get it almost automagically | 20:09 |
| *** EricGonczer_ has quit IRC | 20:09 | |
| fungi | er, if they | 20:09 |
| mtreinish | fungi: if it uses python unittest there is no real barrier | 20:10 |
| *** Goneri has quit IRC | 20:10 | |
| fungi | true | 20:10 |
| *** daemontool has joined #openstack-infra | 20:10 | |
| fungi | this is one of those places where rewriting their ci implementation on their own from scratch has penalized them and cut them out of opportunities to take advantage of tooling we've built on top of our standardized solutions | 20:10 |
| pabelanger | mtreinish: sadly this would be using bash | 20:11 |
| *** coolsvap has quit IRC | 20:11 | |
| mtreinish | pabelanger: heh, then yeah using tempest plugins probably won't work | 20:11 |
| pabelanger | however, I am trying to bring them more inline with openstack-infra tooling, so changing it could be something that needs to be done | 20:12 |
| timothyb89 | pabelanger: you can also generate subunit from bash with subunit-output | 20:12 |
| mtreinish | therve: that or generate-subunit (which is what devstack uses) | 20:12 |
| mtreinish | s/therve/timothyb89 | 20:12 |
| timothyb89 | oh, cool, I hadn't seen that | 20:13 |
| mtreinish | it's basically a much simpler subunit-output, but it supports setting timestamps | 20:14 |
| mtreinish | timothyb89: http://git.openstack.org/cgit/openstack/os-testr/tree/os_testr/generate_subunit.py | 20:14 |
| timothyb89 | ah, yeah, that is a pretty vital feature missing from subunit-output :) | 20:14 |
| mtreinish | timothyb89: https://github.com/testing-cabal/subunit/pull/17 | 20:15 |
| pabelanger | timothyb89: mtreinish: okay great. Looks like I know who to ping with questions | 20:15 |
| bnemec | Doing what devstack does makes sense, since we're doing essentially the same thing as it is. | 20:15 |
| bnemec | Trying to wrap tripleo-ci in Tempest or Python unit tests would be a very weird fit. | 20:15 |
| mtreinish | timothyb89: only open for 8 months :) | 20:16 |
| phschwartz | jeblair: Hey, you around? I have a nodepool issue that I want to talk to you about. | 20:17 |
| phschwartz | jeblair: http://paste.openstack.org/show/547846/ | 20:17 |
| phschwartz | fungi: ^ I haven't found the cause, but found a nodepool issue that has lead to issues for us. You might want to look at that log because you never know if infra will hit the issue. | 20:18 |
| jeblair | phschwartz: howdy | 20:18 |
| jeblair | phschwartz: the issue is a node ran 2 jobs? | 20:19 |
| phschwartz | jeblair: that log I posted shows 2 unique jobs both being run by jenkins on the same node. | 20:19 |
| phschwartz | jeblair: we are seeing it happy in about 40-50% of our runs now | 20:19 |
| jeblair | phschwartz: i don't see OFFLINE_NODE_WHEN_COMPLETE in the parameters list there | 20:19 |
| jeblair | phschwartz: is that not being set by zuul for some jobs? maybe a job regex in zuul's layout.yaml accidentally caused the function that adds that to stop being applied to some jobs? | 20:20 |
| fungi | which is the first thing i looked for in there as well | 20:20 |
| *** ihrachys has quit IRC | 20:20 | |
| phschwartz | jeblair: hmm, I think a change might have caused that to go away. I missed it completely. ty | 20:21 |
| phschwartz | fungi: you as well :) | 20:21 |
| jeblair | np | 20:21 |
| fungi | phschwartz: missing that parameter is just about always the reason for jobs to get rerun on the same nodes. another common (but less frequent) cause is a jenkins master restart causing it to bring nodes back online that haven't been deleted yet | 20:23 |
| *** adrian_otto has quit IRC | 20:24 | |
| phschwartz | fungi: I didn't realize the second. Good to know. nibalizer and I were discussing this earlier and the thought then would be OFFLINE_NODE_WHEN_COMPLETE | 20:24 |
| fungi | yeah, jenkins is really diligent about assuming that nodes are only ever accidentally offline and will try to bring them online again if restarted | 20:25 |
| fungi | so before we ceased using jenkins entirely, as procedure any time we restarted a jenkins master we'd remove slave entries from its config on disk | 20:25 |
| *** eeiden has joined #openstack-infra | 20:26 | |
| fungi | annoying, but did the trick | 20:26 |
| *** HeOS has quit IRC | 20:28 | |
| *** tchaypo has joined #openstack-infra | 20:30 | |
| *** ddieterly is now known as ddieterly[away] | 20:32 | |
| *** mhickey has quit IRC | 20:33 | |
| *** karthik__ has quit IRC | 20:33 | |
| *** Jeffrey4l_ has joined #openstack-infra | 20:35 | |
| *** Jeffrey4l has quit IRC | 20:35 | |
| fungi | mtreinish: 350764 looks good. would also be nice to have the quick one-liner examples of tools you've been using to test its straight mqtt and websockets interfaces | 20:36 |
| fungi | makes it a lot easier for people to check that it's working as intended | 20:36 |
| openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci: Remove test-environments from tripleo-ci https://review.openstack.org/350779 | 20:37 |
| mtreinish | fungi: like the mosquitto_sub command I was using? | 20:38 |
| fungi | mtreinish: yep | 20:39 |
| *** roxanaghe has joined #openstack-infra | 20:39 | |
| fungi | and maybe a sentence mentioning the package names that provide those client tools | 20:39 |
| *** baoli_ has quit IRC | 20:39 | |
| *** tchaypo_vacay has quit IRC | 20:39 | |
| *** ddieterly[away] is now known as ddieterly | 20:40 | |
| *** baoli has joined #openstack-infra | 20:40 | |
| mtreinish | fungi: I can do that, want me to respin or followon patch it | 20:40 |
| fungi | mtreinish: whichever is easier for you. i'm happy to write it too, but that would take research on my part to piece together which you likely already have handy | 20:42 |
| *** csomerville has joined #openstack-infra | 20:42 | |
| *** mctaylor has quit IRC | 20:43 | |
| mtreinish | fungi: well I was probably just gonna link to: https://github.com/mqtt/mqtt.github.io/wiki/libraries | 20:43 |
| *** csomerville has quit IRC | 20:44 | |
| mtreinish | and then just put "mosquitto_sub -h firehose.openstack.org --topic '#'" as an example command | 20:45 |
| fungi | wfm | 20:45 |
| *** csomerville has joined #openstack-infra | 20:45 | |
| fungi | and you had another command for testing the websockets interface right? | 20:45 |
| *** mctaylor has joined #openstack-infra | 20:46 | |
| *** jsavak has quit IRC | 20:46 | |
| mtreinish | fungi: http://mitsuruog.github.io/what-mqtt/ | 20:46 |
| *** cody-somerville has quit IRC | 20:46 | |
| mtreinish | or a python script that uses paho-mqtt to establish a connection over websocket | 20:46 |
| *** jsavak has joined #openstack-infra | 20:46 | |
| *** daemontool has quit IRC | 20:48 | |
| fungi | indeed, the mosquitto-clients package on debian seems to get me a working mosquitto_sub for that | 20:50 |
| mtreinish | fungi: http://paste.openstack.org/show/547853/ | 20:51 |
| mtreinish | if you want to use websocks and not via a random js app hosted on github I found from a google search | 20:51 |
| fungi | heh | 20:52 |
| *** Sukhdev has quit IRC | 20:52 | |
| fungi | but that sounds so fun | 20:52 |
| *** Sukhdev has joined #openstack-infra | 20:53 | |
| fungi | for the what-mqtt page, what does your ws:// url look like? i'm not having a lot of luck guessing | 20:53 |
| *** amitgandhinz has joined #openstack-infra | 20:55 | |
| *** furlongm has joined #openstack-infra | 20:57 | |
| mtreinish | fungi: ws://firehose.openstack.org:80 is what I was using before | 20:57 |
| mtreinish | it still works | 20:57 |
| *** furlongm_ has quit IRC | 20:57 | |
| mtreinish | you've got to hit connect first, then subscribe | 20:57 |
| *** jsavak has quit IRC | 20:58 | |
| fungi | yeah, tried that with # as the topic, similar to your mosquitto_sub example | 20:59 |
| *** acoles_ is now known as acoles | 20:59 | |
| *** amitgandhinz has quit IRC | 21:00 | |
| fungi | might just be my browser is terrible | 21:00 |
| mtreinish | fungi: http://i.imgur.com/o5y3sJr.png | 21:00 |
| mtreinish | fungi: meh, don't worry about it, the python script I pasted should work just as well | 21:01 |
| mtreinish | although I should try running it first :) | 21:01 |
| fungi | ahh, it's never even giving me "connection open" when i try to connect | 21:01 |
| fungi | so could be some browser plugin interfering | 21:01 |
| fungi | maybe privacy badger or something | 21:01 |
| *** matt-borland has quit IRC | 21:03 | |
| fungi | there we go, that was it | 21:04 |
| fungi | eff privacy badger didn't like it opening a websocket to another domain i think | 21:04 |
| mtreinish | fungi: that would make sense. Normally you probably wouldn't want that happening | 21:05 |
| mtreinish | fungi: hmm, so maybe that python script doesn't work. It doesn't seem to actually be establishing a connection for me | 21:05 |
| fungi | running afoul of privacy python, no doubt ;) | 21:06 |
| *** dkranz has quit IRC | 21:06 | |
| mtreinish | fungi: hmm, the script works fine without setting it to be a websocket transport (on the regular mqtt port) | 21:08 |
| *** yamamoto has joined #openstack-infra | 21:09 | |
| mtreinish | fungi: oh, but if I set the transport to websocket and don't set it to port 80 it works fine but with regular mqtt traffic | 21:10 |
| mtreinish | I suspect something is quite right with the library | 21:10 |
| fungi | quite right indeed | 21:10 |
| *** rbuzatu has joined #openstack-infra | 21:10 | |
| fungi | or the not kind of quite right maybe | 21:10 |
| *** jkilpatr has quit IRC | 21:11 | |
| mtreinish | heh, oops. Yeah missed a not there | 21:11 |
| jeblair | pabelanger: http://git.openstack.org/cgit/openstack-infra/project-config/tree/jenkins/jobs/packaging-mirror.yaml#n11 is the only use of ZUUL_COMMIT in project-config.... | 21:11 |
| mtreinish | fungi: haha, it's 'websockets' not websocket' | 21:11 |
| jeblair | pabelanger: how do the files get there in the first place? | 21:11 |
| jeblair | pabelanger: (that usage is downloading from that URL -- where are they uploaded to that path?) | 21:11 |
| mtreinish | fungi: http://paste.openstack.org/show/547856/ works | 21:12 |
| *** ldnunes has quit IRC | 21:13 | |
| mtreinish | I don't know why it doesn't raise an exception when you specify an invalid transport type though | 21:13 |
| mtreinish | that's kinda annoying | 21:13 |
| openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci: Remove test-environments from tripleo-ci https://review.openstack.org/350779 | 21:13 |
| fungi | mtreinish: cool, that's pretty trivial indeed | 21:13 |
| *** rbuzatu has quit IRC | 21:15 | |
| *** fifieldt has quit IRC | 21:15 | |
| pabelanger | jeblair: current is it using the deb-pkgs publisher to upload: http://git.openstack.org/cgit/openstack-infra/project-config/tree/jenkins/jobs/macros.yaml#n921 | 21:15 |
| fungi | ahh, the package on pypi is paho-mqtt | 21:16 |
| *** ddieterly is now known as ddieterly[away] | 21:16 | |
| pabelanger | jeblair: and zigo as logic in his build script to setup the path for publisher | 21:16 |
| jeblair | pabelanger: where's that script? | 21:16 |
| zigo | pabelanger: Hi there! How do I get deb-auto-backports to publish to the right repo? | 21:17 |
| *** kgiusti has left #openstack-infra | 21:17 | |
| pabelanger | jeblair: deb-openstack-pkg-tools currently | 21:17 |
| *** yamamoto has quit IRC | 21:18 | |
| jeblair | pabelanger, zigo: is that script copied to every branch in that repo? | 21:18 |
| openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci: WIP - Implement undercloud upgrade job - Mitaka -> Newton https://review.openstack.org/346995 | 21:18 |
| pabelanger | jeblair: I think only newton today | 21:19 |
| mtreinish | fungi: ah, yeah it's a bit confusing | 21:19 |
| *** EricGonc_ has quit IRC | 21:19 | |
| mtreinish | to make it a bit more confusing it's part of the mosquitto project too :) | 21:19 |
| pabelanger | jeblair: I think we could move that logic into project-config, if you see a problem with it being in deb-openstack-pkg-tools | 21:19 |
| mtreinish | err, or maybe not. It's definitely also part of eclipse | 21:20 |
| mtreinish | I can't keep track | 21:20 |
| *** EricGonczer_ has joined #openstack-infra | 21:21 | |
| zigo | pabelanger: Can you help me to get https://review.openstack.org/#/c/346095/ to publish to the correct backport repo? | 21:22 |
| zigo | pabelanger: That infra patch is the last one before we really cat get going. | 21:22 |
| fungi | mtreinish: yeah, i couldn't find distro packages. closest i have is: golang-eclipse-paho-dev - Go package implementing MQTT client library by Eclipse Paho | 21:22 |
| *** bhunter71 has joined #openstack-infra | 21:22 | |
| pabelanger | zigo: yes, we need to update the reprepro-import job to accept a different dist | 21:23 |
| zigo | pabelanger: Oh, so that's something for you then? Or should I try? | 21:23 |
| fungi | mtreinish: confirmed websockets poc script works for me as well | 21:24 |
| *** gouthamr has quit IRC | 21:24 | |
| pabelanger | zigo: you should be able to use the reprepro-import-debian-openstack-jessie-backports job and it works | 21:24 |
| mtreinish | fungi: I think this is it: https://packages.debian.org/jessie/python-mosquitto | 21:25 |
| *** seldenr has joined #openstack-infra | 21:25 | |
| *** fifieldt has joined #openstack-infra | 21:25 | |
| mtreinish | or maybe not, looking at the file list I don't know what that is | 21:26 |
| pabelanger | jeblair: do you see an issue with the implementation? | 21:26 |
| zigo | pabelanger: How to do this? | 21:27 |
| *** baoli has quit IRC | 21:27 | |
| zigo | pabelanger: Currently, we only have one job for publishing, called deb-pkgs | 21:27 |
| zigo | pabelanger: How is that linked to the post job that import what's pushed to tarballs/packaging-deb ? | 21:28 |
| pabelanger | zigo: I commented on the review, should be the only change you need | 21:29 |
| zigo | pabelanger: Oh, thanks so much ! | 21:30 |
| openstackgerrit | Merged openstack/os-client-config: Add release notes for 1.19.0 release https://review.openstack.org/350672 | 21:31 |
| openstackgerrit | Thomas Goirand proposed openstack-infra/project-config: Add automatic backports https://review.openstack.org/346095 | 21:32 |
| *** matrohon has quit IRC | 21:32 | |
| *** rhallisey has quit IRC | 21:32 | |
| *** _sarob has quit IRC | 21:35 | |
| openstackgerrit | Jay Faulkner proposed openstack-infra/project-config: Fix reversed logic in Ironic jobs https://review.openstack.org/350791 | 21:35 |
| *** sarob has joined #openstack-infra | 21:35 | |
| *** ddieterly[away] is now known as ddieterly | 21:36 | |
| *** Qiming has quit IRC | 21:37 | |
| zzzeek | hey, gerrit is marked as -1 when only a non-voting job fails, all others pass? https://review.openstack.org/#/c/350656/ | 21:38 |
| *** chem has quit IRC | 21:39 | |
| zzzeek | oh. POST_FAILURE, what is that. | 21:39 |
| openstackgerrit | Jay Faulkner proposed openstack-infra/devstack-gate: Explicitly set IRONIC_BUILD_DEPLOY_RAMDISK https://review.openstack.org/350795 | 21:39 |
| *** Qiming has joined #openstack-infra | 21:39 | |
| jeblair | pabelanger: i was considering dropping it in zuulv3... but i *think* i can keep it if it's useful -- is it desirable that the actual commit sha be used, or is it just an arbitrary identifier unique to the build (ie, we could use the zuul ref instead)? | 21:40 |
| *** rwsu has quit IRC | 21:40 | |
| *** thorst has quit IRC | 21:40 | |
| jeblair | zzzeek: something went wrong uploading the artifacts (eg build log, tarball, etc) after the main part of the job succeeded | 21:40 |
| openstackgerrit | Jay Faulkner proposed openstack-infra/devstack-gate: Explicitly set IRONIC_BUILD_DEPLOY_RAMDISK https://review.openstack.org/350795 | 21:41 |
| zzzeek | jeblair: K so doing a 'recheck' | 21:41 |
| jeblair | speaking of which -- pabelanger should we start upgrading the launchers? | 21:41 |
| jeblair | zzzeek: yep. there is a change in the newest release of ansible we hope will eliminate, or reduce, or at least illuminate the underlying problem | 21:41 |
| zzzeek | +1 for illumination | 21:42 |
| jeblair | pabelanger: do you want me to do a smoke test of the new ansible with my local zuul-launcher setup? | 21:43 |
| *** sdake has quit IRC | 21:43 | |
| *** mctaylor has quit IRC | 21:44 | |
| JayF | I'd apprecaite a review on https://review.openstack.org/350791 as it fixes an Ironic gate issue where we build deploy ramdisks for every job, causing more unneccessary random failures + eating up more gate resources than needed | 21:45 |
| pabelanger | jeblair: I am happy to update our script to use something else, sha1 was the first suggestion we talked in Austin about it. If you think zuul_ref is better, lets do that | 21:45 |
| pabelanger | jeblair: sure, if you'd like to do the test | 21:46 |
| jeblair | pabelanger: okay, let me think about the zuul_commit thing some more | 21:46 |
| jeblair | pabelanger: i'll run the test in a min. | 21:46 |
| jeblair | JayF: that change appears only to remove whitespace | 21:47 |
| JayF | jeblair: egad | 21:47 |
| JayF | jeblair: 1s | 21:48 |
| openstackgerrit | Jay Faulkner proposed openstack-infra/project-config: Fix reversed logic in Ironic jobs https://review.openstack.org/350791 | 21:48 |
| JayF | jeblair: ^ fixed the change, sorry about that | 21:48 |
| jeblair | JayF: you're not using that programming language where the tokens are just spaces and tabs, are you? :) | 21:48 |
| JayF | jeblair: no, I'm using a faulty brain which sometimes swaps values twice, making them the same, lol | 21:49 |
| *** shamail has quit IRC | 21:49 | |
| JayF | jeblair: I'm not convinced that's a good fix anymore, going to -W it and dig in again, make sure I fully understand | 21:52 |
| JayF | something is rotten in that project-config flow and I can't quite find it :( | 21:52 |
| *** vvalderrv has left #openstack-infra | 21:52 | |
| *** burgerk_ has joined #openstack-infra | 21:53 | |
| *** sarob has quit IRC | 21:53 | |
| *** sarob has joined #openstack-infra | 21:54 | |
| *** rockyg has quit IRC | 21:55 | |
| *** amitgandhinz has joined #openstack-infra | 21:56 | |
| *** burgerk has quit IRC | 21:56 | |
| *** burgerk_ has quit IRC | 21:58 | |
| *** markvoelker has quit IRC | 21:58 | |
| *** markvoelker has joined #openstack-infra | 21:59 | |
| jeblair | pabelanger: 2.1.1.0 wfm | 21:59 |
| *** rlandy has quit IRC | 21:59 | |
| *** esberglu has quit IRC | 22:00 | |
| *** amitgandhinz has quit IRC | 22:00 | |
| *** sarob has quit IRC | 22:01 | |
| *** mriedem has quit IRC | 22:01 | |
| *** eharney has quit IRC | 22:02 | |
| *** rwsu has joined #openstack-infra | 22:03 | |
| *** florianf has quit IRC | 22:03 | |
| *** eharney has joined #openstack-infra | 22:03 | |
| *** thorst has joined #openstack-infra | 22:03 | |
| *** markvoelker has quit IRC | 22:03 | |
| *** mctaylor has joined #openstack-infra | 22:04 | |
| *** signed8bit is now known as signed8bit_Zzz | 22:04 | |
| *** mctaylor has quit IRC | 22:04 | |
| *** mctaylor has joined #openstack-infra | 22:05 | |
| *** piet has quit IRC | 22:05 | |
| pabelanger | jeblair: great, ready here. 349103 needs +A | 22:05 |
| *** adrian_otto has joined #openstack-infra | 22:06 | |
| *** mdrabe_ has quit IRC | 22:07 | |
| jeblair | pabelanger: done | 22:07 |
| *** thorst has quit IRC | 22:07 | |
| *** mmandel has quit IRC | 22:08 | |
| *** acoles is now known as acoles_ | 22:09 | |
| openstackgerrit | Jay Faulkner proposed openstack-infra/project-config: Forbid ironic grenade job from building ramdisk https://review.openstack.org/350791 | 22:09 |
| JayF | jeblair: ^ that fixes the real issue, sorry for getting it wrong the first time | 22:10 |
| JayF | What's supposed to be the proper interaction between project-config and devstack-gate? Like I'm not sure why that intermediate DEVSTACK_GATE_IRONIC_BUILD_RAMDISK exists at all, vs just setting it in devstack configs through the gate | 22:10 |
| *** rbuzatu has joined #openstack-infra | 22:12 | |
| openstackgerrit | K Jonathan Harker proposed openstack-infra/project-config: Add integration tests between system-config and logstash-filters https://review.openstack.org/320729 | 22:12 |
| *** rwsu has quit IRC | 22:13 | |
| *** annegentle has quit IRC | 22:13 | |
| *** tphummel has quit IRC | 22:14 | |
| *** signed8bit_Zzz is now known as signed8bit | 22:16 | |
| *** adrian_otto has quit IRC | 22:16 | |
| *** rbuzatu has quit IRC | 22:17 | |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul: Ansible launcher: re-register functions after disconnect https://review.openstack.org/350807 | 22:17 |
| mtreinish | fungi: were you working on the doc patch for the client side mqtt stuff? If not I can write something up in a bit | 22:18 |
| jeblair | pabelanger: we should probably merge that -- it's important, ^ but i'm not sure we need to hold up the current ansible-upgrade restart for it. | 22:18 |
| *** signed8bit has quit IRC | 22:18 | |
| openstackgerrit | Merged openstack-infra/puppet-zuul: Upgrade to ansible 2.1.1.0 https://review.openstack.org/349103 | 22:18 |
| pabelanger | jeblair: ack | 22:20 |
| *** dimtruck is now known as zz_dimtruck | 22:20 | |
| jeblair | JayF: i think there's an effort to batch up some of those pass-through variables, though i don't have further details at hand... sdague may know? | 22:20 |
| jeblair | JayF: (but yes, in principle, d-g really only needs to know the ones it needs to know or change via its branch-selection logic) | 22:21 |
| mtreinish | jeblair: well we want to move a lot of it to just write directly to the config files | 22:24 |
| *** rwsu has joined #openstack-infra | 22:24 | |
| mtreinish | jeblair: sdague has a series up for using a local.conf to enable that: https://review.openstack.org/#/q/status:open+project:openstack-infra/devstack-gate+branch:master+topic:local.conf | 22:24 |
| mtreinish | which has been sitting for a while | 22:24 |
| mtreinish | a lot of those intermediate variables in d-g predate the existence of the devstack passthrough var being in devstack-gate | 22:25 |
| openstackgerrit | Jay Faulkner proposed openstack-infra/devstack-gate: Explicitly set IRONIC_BUILD_DEPLOY_RAMDISK https://review.openstack.org/350795 | 22:26 |
| jeblair | mtreinish: yeah, they originated when external configuration of devstack other than what d-g explicitly provided was not a thing (during the time of 'core projects' :) | 22:27 |
| JayF | mtreinish: will put this here as well; I'll file a bug in Ironic for that to get deprecated out of devstack in this case; I just lost an hour+ tracking down gate weirdness b/c I didn't know how all the misdirection worked :) | 22:27 |
| JayF | (although now I know, which is nice) | 22:28 |
| *** yamamoto has joined #openstack-infra | 22:29 | |
| *** gordc has quit IRC | 22:29 | |
| *** rfolco has quit IRC | 22:30 | |
| *** ddieterly has quit IRC | 22:31 | |
| *** yamamoto_ has joined #openstack-infra | 22:31 | |
| *** yamamoto has quit IRC | 22:31 | |
| *** ddieterly has joined #openstack-infra | 22:32 | |
| fungi | mtreinish: happy to write it up, but probably won't be until tomorrow | 22:33 |
| *** karthik__ has joined #openstack-infra | 22:34 | |
| mtreinish | fungi: ok cool, thanks. I probably wouldn't get to it any sooner either | 22:34 |
| *** ddieterly has quit IRC | 22:35 | |
| *** julim has joined #openstack-infra | 22:37 | |
| *** Teresita-Warrior has quit IRC | 22:38 | |
| *** rwsu has quit IRC | 22:40 | |
| pabelanger | jeblair: looks like zlstatic01.o.o is upgrade. Did we want to start with that? | 22:40 |
| jeblair | pabelanger: ++ | 22:41 |
| jeblair | pabelanger: i'm watching logs; you want to hit the button? | 22:41 |
| *** kien-ha has joined #openstack-infra | 22:43 | |
| pabelanger | jeblair: stopped now | 22:43 |
| pabelanger | and starting | 22:43 |
| pabelanger | jeblair: actually, do we need to restart zuul-launcher if ansible-playbook is launched as a process? | 22:44 |
| *** cody-somerville has joined #openstack-infra | 22:44 | |
| jeblair | pabelanger: erm, i think you're right, we probably don't | 22:44 |
| pabelanger | Ya, I see errors in zlstatic01.o.o about ansible-playbook missing | 22:45 |
| pabelanger | which pip uninstalled | 22:45 |
| pabelanger | in that case, we just monitor now | 22:45 |
| *** rwsu has joined #openstack-infra | 22:45 | |
| *** zul_ has quit IRC | 22:46 | |
| jeblair | so zuul will probably relaunch a bunch of jobs because of that | 22:46 |
| pabelanger | Ya, I can see that in grafana | 22:46 |
| *** csomerville has quit IRC | 22:47 | |
| pabelanger | not so rolling upgrades | 22:47 |
| *** mctaylor has quit IRC | 22:47 | |
| jhesketh | Morning | 22:48 |
| *** zz_dimtruck is now known as dimtruck | 22:48 | |
| pabelanger | o/ | 22:48 |
| pabelanger | jhesketh: we just upgrade zuul-launcher to ansible 2.1.1.0 | 22:48 |
| pabelanger | upgraded | 22:49 |
| jhesketh | pabelanger: oh neat :-) | 22:49 |
| *** thiagop has quit IRC | 22:51 | |
| craige | o/ | 22:52 |
| jeblair | jhesketh: is this patch which i accidentally self-approved (sorry!) what you had in mind in your earlier review comment? https://review.openstack.org/350165 | 22:53 |
| jhesketh | jeblair: without having read it in detail, that looks like a good improvement :-) | 22:54 |
| *** EricGonczer_ has quit IRC | 22:54 | |
| jhesketh | jeblair: the less we fake out the better, and just tidying up runAnsible seems like a great point to poke at things | 22:55 |
| jeblair | cool | 22:56 |
| *** xyang1 has quit IRC | 22:56 | |
| *** amitgandhinz has joined #openstack-infra | 22:56 | |
| *** asettle has joined #openstack-infra | 22:57 | |
| jeblair | pabelanger: i'm guessing the nodepool spike is the ansible upgrade (zuul fast-aborting and re-launching jobs when ansible was broken) | 22:59 |
| jeblair | pabelanger: it's returned to normal now which is good | 22:59 |
| pabelanger | jeblair: Yes, I agree with that | 23:00 |
| *** amitgandhinz has quit IRC | 23:01 | |
| *** sdake has joined #openstack-infra | 23:03 | |
| *** seldenr has quit IRC | 23:07 | |
| *** sarob has joined #openstack-infra | 23:07 | |
| *** tpsilva has quit IRC | 23:08 | |
| *** asettle has quit IRC | 23:09 | |
| *** PollyZ has quit IRC | 23:09 | |
| *** pt_15 has quit IRC | 23:10 | |
| *** sarob has quit IRC | 23:11 | |
| *** rbrndt has quit IRC | 23:13 | |
| *** pt_15 has joined #openstack-infra | 23:14 | |
| JayF | We just had an Ironic job fail, with seemingly nothing wrong in it: http://logs.openstack.org/79/350679/4/check/gate-ironic-python35-db/68a5156/console.html#_2016-08-03_22_44_04_407328 | 23:17 |
| JayF | no tests failed, even seeing Task exit code: 0, but at the end is result: FAILURE | 23:17 |
| pabelanger | jeblair: possible ansible issue^ | 23:19 |
| pabelanger | http://logs.openstack.org/79/350679/4/check/gate-ironic-python35-db/68a5156/_zuul_ansible/ansible_log.txt | 23:19 |
| pabelanger | The async task did not return valid JSON: expected string or buffer | 23:19 |
| pabelanger | JayF: we just upgraded ansible on zuul-launchers, so this could be a side affect | 23:19 |
| JayF | https://review.openstack.org/#/c/350679/ this change is the one, a passing example + 2 failing examples | 23:20 |
| JayF | (the pep8 failure is real | 23:20 |
| openstackgerrit | Craige McWhirter proposed openstack-infra/infra-specs: Initial commit of the Pholio spec https://review.openstack.org/340641 | 23:20 |
| pabelanger | JayF: try a recheck | 23:21 |
| pabelanger | JayF: I am checking out logs to see when we installed ansible 2.1.1.0 | 23:21 |
| pabelanger | Aug 3 22:40:40 zl01 puppet-user[11566]: (/Stage[main]/Zuul::Launcher/Package[ansible]/ensure) ensure changed '2.1.0.0' to '2.1.1.0' | 23:22 |
| pabelanger | I wonder if the running ansible-playbook process, which was 2.1.0.0, started using the bits from the 2.1.1.0 install which caused the issue | 23:24 |
| JayF | I'm going to fix the pep8 issue on that patch and repush | 23:24 |
| pabelanger | 2016-08-03 22:44:03,713 p=19990 u=zuul | fatal: [node]: FAILED! => {"changed": false, "failed": true, "msg": "The async task did not return valid JSON: expected string or buffer"} | 23:25 |
| pabelanger | the failure is 4mins after ansible was upgraded | 23:25 |
| pabelanger | so, timeframe matches up | 23:25 |
| openstackgerrit | Jay Faulkner proposed openstack-infra/project-config: Forbid ironic grenade job from building ramdisk https://review.openstack.org/350791 | 23:26 |
| *** sdague has quit IRC | 23:29 | |
| *** mrmartin has quit IRC | 23:33 | |
| *** tonytan4ever has quit IRC | 23:33 | |
| *** fernnest_ has quit IRC | 23:34 | |
| *** dingyichen has joined #openstack-infra | 23:34 | |
| *** fernnest has quit IRC | 23:35 | |
| osnexus | If i have a devstack environment, how would I go about trying to make sure I can ssh into compute nodes? It seems I can only see the network node neutron spins up, but not its compute nodes | 23:37 |
| docaedo | osnexus: devstack is an all-in-one openstack environment, so the VM you run it on has all the services there | 23:41 |
| *** Swami has quit IRC | 23:41 | |
| osnexus | docaedo right, but I can't see what the services are creating. I'm trying to use it as part of a 3rd party CI system, but the CI node needs to get at the compute nodes created by devstack | 23:43 |
| osnexus | But nobody can see them | 23:43 |
| osnexus | except for the one neutron network node | 23:43 |
| openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci: WIP - Implement undercloud upgrade job - Mitaka -> Newton https://review.openstack.org/346995 | 23:44 |
| docaedo | when you say "compute nodes created by devstack" do you mean VMs devstack launches? | 23:44 |
| *** zhurong has joined #openstack-infra | 23:45 | |
| *** _ari_ has joined #openstack-infra | 23:45 | |
| osnexus | docaedo yes, nodepool and jenkins on the CI VM send images over to the devstack VM which then spins them up | 23:46 |
| osnexus | As VMs inside itself I believe | 23:46 |
| osnexus | VM inside a VM | 23:46 |
| osnexus | correct? | 23:46 |
| jeblair | pabelanger, JayF: well that's the error than we hope the ansible upgrade will fix, so it's also possible that was just one more incidence of the error and proximity to the upgrade event is a coincidence. | 23:47 |
| docaedo | osnexus: ok, I kind of follow - but I'll leave third party CI debugging questions to the experts :) I thought maybe it was just a simple devstack question but I think this is a little more complicated | 23:47 |
| jeblair | mordred: https://review.openstack.org/349037 is in merge conflict | 23:47 |
| *** roxanaghe has quit IRC | 23:47 | |
| *** jamielennox is now known as jamielennox|away | 23:48 | |
| *** _ari_ has quit IRC | 23:48 | |
| osnexus | ya.... the complication never ends on this.... | 23:49 |
| JayF | jeblair: we had two tests fail on the same patch in that way, so it's either an impact or we're very, very unlucky. I buy either explanation, and it doesn't matter anyway. The best kinds of bugs to discover are those someone else already fixed for you \o/ | 23:51 |
| *** sdake has quit IRC | 23:51 | |
| *** zhurong has quit IRC | 23:54 | |
| docaedo | osnexus: don't know if this will help, but if devstack neutron uses provider network, VMs can be on the same network as devstack itself (which would reduce routing complexity at least), then make sure security groups are configured to allow traffic in to the vms | 23:54 |
| *** sdake has joined #openstack-infra | 23:55 | |
| *** amitgandhinz has joined #openstack-infra | 23:57 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!
