*** ^d has quit IRC | 00:01 | |
jog0 | sdague clarkb: we are getting build timeout after 90 minutes | 00:01 |
---|---|---|
jog0 | in {check,gate}-temepst-dsvm-{,postgres}-full | 00:02 |
anteaya | yeah, I noticed those had jumped up | 00:04 |
anteaya | what might cause the sudden increase in those bugs? | 00:04 |
jog0 | hmm its because suspend and resume and reboot tests are failing in nova | 00:04 |
jog0 | I think | 00:04 |
jog0 | so if things timeout we don't collect logs :( | 00:05 |
*** markmcclain has joined #openstack-infra | 00:05 | |
jog0 | jeblair: ^ | 00:05 |
jog0 | err testr doesn't save logs or something | 00:05 |
jog0 | lifeless: ^ | 00:06 |
jeblair | jog0: if jenkins times out logs are not copied to logs.o.o | 00:06 |
jeblair | jog0: two solutions: try to make the jobs timeout internally (eg testr) before hitting the jenkins timeout | 00:06 |
lifeless | jog0: logs are written to a temp file in real time | 00:06 |
jog0 | jeblair: some appear t o be http://logs.openstack.org/90/54290/4/check/check-tempest-dsvm-full/d03c4f1/console.html#_2013-12-17_19_51_29_703 | 00:06 |
jog0 | jeblair: I like the testr timeout option | 00:07 |
lifeless | jog0: so should be able to just grab them | 00:07 |
jog0 | lifeless: is that doable, a global 90 minute timeout ? | 00:07 |
jog0 | for tempest | 00:07 |
jog0 | or we can wire up logic to look for those temp files | 00:07 |
lifeless | sure, new feature | 00:07 |
lifeless | shortest path is to grab the temp file | 00:07 |
jog0 | where do they live? | 00:07 |
lifeless | .testrepository/ | 00:08 |
jeblair | jog0: those logs were put in position before the test run started... | 00:08 |
jeblair | jog0: you want the testr logs or the openstack logs? | 00:08 |
*** jhesketh__ has quit IRC | 00:08 | |
jog0 | jeblair: testr for starters | 00:08 |
jog0 | ideally both openstack and testr | 00:08 |
*** sarob has joined #openstack-infra | 00:08 | |
jog0 | but one or the other is pretty good | 00:08 |
jeblair | jog0: the openstack logs are moved into position by devstack-gate for jenkins to find after tempest finishes | 00:09 |
jog0 | I have some failures in tempest here http://logs.openstack.org/90/54290/4/check/check-tempest-dsvm-full/d03c4f1/console.html#_2013-12-17_18_48_58_247 | 00:09 |
jog0 | and want to get some more info on them | 00:09 |
jeblair | jog0: perhaps we could replace the tempest invocation with "timeout 85m tox -e full" in devstack-gate | 00:10 |
jog0 | lifeless: ^ thoughts | 00:10 |
jeblair | jog0: and use that to kill tempest before the jenkins timeout while giving devstack-gate enough time to copy the logs into position for jenkins | 00:10 |
jog0 | I think we loose the testr logs taht way, but we get openstack logs, so win | 00:11 |
jeblair | jog0: if they are written in real time we should get whatever is run | 00:12 |
jog0 | jeblair: we have to add logic to copy the temp file Ithink | 00:12 |
*** sarob has quit IRC | 00:13 | |
jog0 | oh this should n | 00:13 |
jog0 | patch coming soon | 00:13 |
*** rnirmal has joined #openstack-infra | 00:13 | |
lifeless | there is some buffering in the logs atm | 00:13 |
lifeless | once I finish the migration to v2 it will be gone | 00:13 |
lifeless | or so low as to be invisible | 00:13 |
lifeless | specifically you won't see hung tests today, only slow ones | 00:13 |
openstackgerrit | James E. Blair proposed a change to openstack-infra/devstack-gate: WIP: test timing out tempest https://review.openstack.org/62781 | 00:14 |
jog0 | lifeless: that is fine, | 00:14 |
jeblair | jog0: ^ quick poc of the idea | 00:14 |
*** cody-somerville has joined #openstack-infra | 00:14 | |
*** cody-somerville has quit IRC | 00:14 | |
*** cody-somerville has joined #openstack-infra | 00:14 | |
jeblair | jog0: if it works and we do it for real, we'll probably want to pass in the timeout values from the JJB config so that we can accomodate the different runtimes for different jobs | 00:15 |
cody-somerville | clarkb: Hey. Where are you guys? | 00:15 |
pleia2 | cody-somerville: clarkb went to the airport, the rest of us are in the gravity room on the 7th floor | 00:16 |
cody-somerville | Ah, ok. | 00:16 |
*** rongze has joined #openstack-infra | 00:16 | |
cody-somerville | pleia2: Who all is up there? | 00:16 |
jeblair | pleia2: sounds heavy. | 00:17 |
*** mrodden has quit IRC | 00:17 | |
StevenK | jeblair: Boo. Hiss. | 00:17 |
pleia2 | cody-somerville: AaronGr and I with the hp folks | 00:17 |
*** herndon has quit IRC | 00:17 | |
pleia2 | er, other hp folks, non-monty hp folks :) | 00:17 |
pleia2 | jeblair: hehe | 00:18 |
jog0 | lifeless jeblair: http://paste.openstack.org/show/55226 | 00:18 |
fungi | "heavy." why are things so heavy in the future? | 00:18 |
jog0 | something like that would let us colelct the testr temp file I think | 00:19 |
*** loq_mac has quit IRC | 00:20 | |
ewindisch | in SF - anyone up for beer, coffee, or dinner? | 00:20 |
jeblair | jog0: the testr part looks good (the timeout we'll want to make less hacky like above) | 00:20 |
openstackgerrit | A change was merged to openstack/requirements: Add python-ironicclient to global requirements https://review.openstack.org/60638 | 00:21 |
*** rongze has quit IRC | 00:21 | |
jog0 | jeblair: testing the idea out locally using unit tests | 00:21 |
pleia2 | ewindisch: aww, I'm in mt view tonight, how long are you in town? | 00:22 |
jeblair | jog0: oh, and the other thing is we need to budget for setup time; so the tempest timeout should be (job_timeout - devstack_setup_time - log_prep_time - margin_of_error) | 00:22 |
jeblair | jog0: oh, i had another idea... | 00:22 |
jog0 | jeblair: cool, I am making sure the testr logic works for now | 00:23 |
jeblair | jog0: devstack and tempest are all in the gate script, whereas log copying is in the wrap script... so we may be able to timeout the gate script | 00:23 |
jog0 | looks like it does | 00:23 |
jeblair | jog0: which makes the timeout calculation less fiddly | 00:24 |
jog0 | jeblair: oh I like | 00:24 |
ewindisch | pleia2: until Thursday, but I have something Thursday night | 00:24 |
lifeless | +1 | 00:24 |
jog0 | jeblair: confirmed timeout will make testr leave its temp file around | 00:24 |
jog0 | going to push a patch that does only the testr copy logic | 00:24 |
openstackgerrit | James E. Blair proposed a change to openstack-infra/devstack-gate: WIP: test timout #2 https://review.openstack.org/62784 | 00:25 |
jeblair | jog0: testing the other timeout idea | 00:25 |
jeblair | jog0: ++copy logic patch | 00:25 |
jeblair | jog0: 5m tempest timeout seems to have worked: https://jenkins01.openstack.org/job/check-dg-tempest-dsvm-full/30/console | 00:29 |
cody-somerville | If I want to write a new config module and sort of trail and error a bit on a cloud instance, is there a particular version of puppet I should be using? | 00:29 |
openstackgerrit | Joe Gordon proposed a change to openstack-infra/devstack-gate: Preserve testr temp files https://review.openstack.org/62786 | 00:29 |
jeblair | cody-somerville: we're using 2.7.21 from puppetlabs on prod servers | 00:29 |
jog0 | jeblair: cool I did a 1m timeout locally and saw the same thing | 00:30 |
jog0 | ohh nice | 00:30 |
jeblair | cody-somerville: http://ci.openstack.org/sysadmin.html#making-a-change-in-puppet | 00:30 |
jeblair | cody-somerville: if you follow the instructions there, you'll get exactly the puppet version we use in prod | 00:30 |
*** dizquierdo has quit IRC | 00:31 | |
jog0 | lifeless: https://review.openstack.org/#/c/62786/ | 00:32 |
cody-somerville | jeblair: Awesome. Thanks. | 00:33 |
openstackgerrit | Joe Gordon proposed a change to openstack-infra/devstack-gate: Preserve testr temp files https://review.openstack.org/62786 | 00:33 |
openstackgerrit | James E. Blair proposed a change to openstack-infra/devstack-gate: WIP: test timout #2 https://review.openstack.org/62784 | 00:36 |
*** wenlock has joined #openstack-infra | 00:38 | |
jeblair | jog0: 5m timeout of the gate script correctly killed during devstack, though parts of devstack kept running in the bg... | 00:38 |
jeblair | https://jenkins02.openstack.org/job/check-dg-tempest-dsvm-full/26/console | 00:39 |
jog0 | jeblair: is that a problem? | 00:39 |
jog0 | its clearly not good | 00:39 |
jeblair | jog0: i don't think so? it's not very elegant but i don't think it'll happen in practice and i think we'll still get the bulk of the data | 00:40 |
jeblair | jog0: tox and testr seemed better behaved so i don't think this case is worth worrying about too much | 00:40 |
jog0 | agreed | 00:40 |
openstackgerrit | James E. Blair proposed a change to openstack-infra/config: Refactor logstash_worker into log_processor module https://review.openstack.org/62787 | 00:41 |
fungi | the gate is empty, so as soon as there are no queued jobs in the post pipeline (one hasn't been started yet) i'll put jenkins.o.o in prepare for shutdown | 00:42 |
fungi | and there it goes | 00:42 |
fungi | once the rest of those finish, i'll switch the dns entry, fire off one last catch-up rsync and update the root path for the tarballs.o.o publisher site, wait ~5 minutes and cancel the jenkins shutdown | 00:43 |
fungi | actually as soon as the last running tarball job there finishes, i'll do the rsync | 00:44 |
anteaya | pleia2: so is the onboarding continuing tomorrow? | 00:44 |
*** wenlock has quit IRC | 00:44 | |
anteaya | pleia2: I still need to hear from mordred otherwise I will do what I have to do and just accept the fingerwagging when it happens | 00:44 |
openstackgerrit | James E. Blair proposed a change to openstack-infra/config: Increase to 16 logstash worker nodes https://review.openstack.org/62788 | 00:44 |
*** jergerber has quit IRC | 00:45 | |
jeblair | fungi: ^ the first of those logstash changes should let us double-up the workers on the nodes, and then the second should let us scale up | 00:45 |
fungi | exciting--having a look | 00:45 |
pleia2 | anteaya: no, folks are already heading home | 00:46 |
jeblair | fungi: we should probably do both of those and then give crm114 another shot; keeping an eye on it. then hopefully we can scale down. | 00:46 |
anteaya | pleia2: k, thanks - I hope everyone had a good time | 00:46 |
fungi | final tarballs rsync done | 00:46 |
anteaya | and that mordred reads my email | 00:46 |
anteaya | one day left to wait | 00:46 |
* anteaya makes pointy fingers at mordred | 00:47 | |
fungi | i think it's probably safe to let the requirements post mirror jobs keep running while i cut over the tarballs site | 00:47 |
jeblair | fungi: yes | 00:47 |
*** yamahata has joined #openstack-infra | 00:47 | |
pleia2 | anteaya: we split up into teams today, I haven't actually seen mordred since last night, I'll let him know you have a query pending if I do | 00:48 |
*** zhiyan has joined #openstack-infra | 00:48 | |
clarkb | double glad I left a little early screening line is huge | 00:49 |
fungi | jeblair: do you think the jvm name resolver jenkins uses will notice the address change automatically, or should i wait until it's fully quiesced and completely restart the jenkins service for good measure? | 00:50 |
StevenK | clarkb: The TSA are just being extra careful. | 00:50 |
StevenK | Wait, I can't claim that with a straight face. | 00:50 |
jeblair | fungi: jenkins seems to handle it correctly, so i don't think it's necessary | 00:50 |
fungi | StevenK: extra careful to jusitfy their pay checks | 00:50 |
fungi | jeblair: okay, good enough. a few more minutes and i'll cancel shutdown then | 00:51 |
fungi | and i'll do some spot checks after the next tarball job i see complete, just to make sure it's landing on the correct server | 00:51 |
jog0 | jeblair: this is the related bug https://bugs.launchpad.net/tempest/+bug/1258682 | 00:52 |
uvirtbot | Launchpad bug 1258682 in tempest "timeout causing gate-tempest-dsvm-full to fail" [Undecided,Invalid] | 00:52 |
jog0 | the generic things timed out and I don't have logs | 00:52 |
fungi | jeblair: i ended up adding the additional filesystem graph through the cacti admin webui. hopefully that doesn't cause issues | 00:52 |
fungi | (couldn't find a sane way to inject the info into the db) | 00:53 |
jeblair | fungi: oh, yeah, that's probably the best way to deal with a one-off | 00:53 |
anteaya | pleia2: sure, clarkb was my messenger yesterday, so a smile and reminder should do it | 00:53 |
anteaya | it's just I only have one day left | 00:54 |
anteaya | otherwise I could wait | 00:54 |
*** ^d has joined #openstack-infra | 00:57 | |
*** nicedice has quit IRC | 00:57 | |
*** nicedice has joined #openstack-infra | 00:57 | |
*** johnthetubaguy has joined #openstack-infra | 01:00 | |
*** esker has joined #openstack-infra | 01:01 | |
*** ^d has quit IRC | 01:02 | |
*** wenlock has joined #openstack-infra | 01:02 | |
fungi | just dawned on me that it's jenkins01 and jenkins02 which do the tarball builds, so i quickly ninja'd the tarballs.o.o site root path on them just now. no new tarball jobs have started yet anyway | 01:03 |
*** blamar has quit IRC | 01:03 | |
*** johnthetubaguy has quit IRC | 01:04 | |
fungi | in fact, 90% of our developers seem to have decided to go on holiday this week, judging from the gate | 01:06 |
jerryz | jog0: does grenade failure log get classified if tempests are successful? | 01:06 |
jog0 | jerryz: we don't report on grenade failures at the moment | 01:07 |
jerryz | jog0: but it seems to me that if tempest and grenade fail at the same time, grenade failures will still get reported, as grenade logs are in elastic search | 01:08 |
jog0 | jerryz: we may have changed it we wantto report on grenade | 01:09 |
*** praneshp has joined #openstack-infra | 01:11 | |
jerryz | jog0: i mean i found that tempest failure triggered the classification, but in the end, as a matter of fact, a bug is found in grenade logs . | 01:12 |
*** wenlock has quit IRC | 01:12 | |
jog0 | jerryz: that sounds possible | 01:13 |
*** praneshp_ has joined #openstack-infra | 01:13 | |
*** praneshp has quit IRC | 01:15 | |
*** praneshp_ is now known as praneshp | 01:15 | |
jerryz | jog0: that happens when i fight the grenade failures. i found Bug 1253896 failed my grenade test a lot | 01:15 |
uvirtbot | Launchpad bug 1253896 in neutron "Attempts to verify guests are running via SSH fails. SSH connection to guest does not work." [Critical,In progress] https://launchpad.net/bugs/1253896 | 01:15 |
fungi | that one seems vague enough to be a symptom of more than one actual bug | 01:16 |
jog0 | yeah we don't know much about what is causing that bug | 01:17 |
jerryz | fungi, jog0: in my env, 3 rechecks may give me luck to pass. | 01:19 |
*** senk1 has quit IRC | 01:20 | |
*** freyes_ has joined #openstack-infra | 01:21 | |
*** arosen has quit IRC | 01:22 | |
*** jhesketh__ has joined #openstack-infra | 01:23 | |
jerryz | jog0: why not report grenade failures at the moment? | 01:23 |
jog0 | jerryz: its a bug | 01:23 |
jerryz | jog0: return "FAILURE" in line and "tempest" in line . can i add 'or "grenade" in line' here | 01:25 |
*** blamar has joined #openstack-infra | 01:26 | |
*** nosnos has joined #openstack-infra | 01:26 | |
jerryz | jog0: http://git.openstack.org/cgit/openstack-infra/elastic-recheck/tree/elastic_recheck/elasticRecheck.py#n82 | 01:29 |
*** yaguang has joined #openstack-infra | 01:29 | |
jog0 | jerryz: that isn't enough | 01:30 |
jog0 | jerryz: can you find an example where 1253896 is failing greande | 01:30 |
jog0 | grenade | 01:30 |
jog0 | grenade has different files | 01:30 |
jeblair | fungi: gah, sorry about that (tarball jobs) | 01:31 |
fungi | jeblair: no damage done. it's quiet in our gated community tonight | 01:32 |
fungi | i think we're coming up on an hour straight with nothing at all in the gate | 01:32 |
jeblair | that is a very flat sparkline! | 01:33 |
jerryz | jog0: http://logstash.openstack.org/#eyJzZWFyY2giOiJtZXNzYWdlOiBcIkJhZFJlcXVlc3Q6IEludmFsaWQgdm9sdW1lOiBWb2x1bWUgc3RhdHVzIG11c3QgYmUgYXZhaWxhYmxlIG9yIGVycm9yXCIgQU5EIGZpbGVuYW1lOiBcImNvbnNvbGUuaHRtbFwiIEFORCBidWlsZF9uYW1lOiBcImNoZWNrLWdyZW5hZGUtZHN2bVwiIiwiZmllbGRzIjpbXSwib2Zmc2V0IjowLCJ0aW1lZnJhbWUiOiI2MDQ4MDAiLCJncmFwaG1vZGUiOiJjb3VudCIsInRpbWUiOnsidXNlcl9pbnRlcnZhbCI6MH0sInN0YW1wIjoxMzg3MzMwNDUxMTg0fQ== | 01:35 |
*** senk1 has joined #openstack-infra | 01:36 | |
*** crank has quit IRC | 01:38 | |
*** crank has joined #openstack-infra | 01:38 | |
openstackgerrit | Joe Gordon proposed a change to openstack-infra/elastic-recheck: Add support for grenade console.html https://review.openstack.org/62791 | 01:40 |
jog0 | jerryz: ^ that should support your fingerprint | 01:40 |
jog0 | sdague: ^ | 01:40 |
jog0 | jerryz: I need to go AFK for a bit | 01:40 |
jog0 | o/ | 01:40 |
jerryz | thanks | 01:40 |
openstackgerrit | James E. Blair proposed a change to openstack-infra/devstack-gate: WIP: test timout #2 https://review.openstack.org/62784 | 01:41 |
*** CaptTofu has quit IRC | 01:42 | |
*** CaptTofu has joined #openstack-infra | 01:42 | |
*** ryanpetrello has joined #openstack-infra | 01:42 | |
*** sdake has quit IRC | 01:42 | |
*** sdake has joined #openstack-infra | 01:42 | |
*** sdake has quit IRC | 01:43 | |
*** sdake has joined #openstack-infra | 01:43 | |
*** cody-somerville has quit IRC | 01:48 | |
*** crank has quit IRC | 01:50 | |
*** senk1 has quit IRC | 01:50 | |
*** crank has joined #openstack-infra | 01:51 | |
fungi | changes in the gate finally. soon i'll be able to confirm tarball uploads are working | 02:00 |
fungi | jhesketh_ really should have added tarball jobs for turbo-hipster. boo | 02:00 |
fungi | have to wait for the nova change to (hopefully) merge instead | 02:01 |
jhesketh__ | fungi: yes, I need to neaten up the release stuff for that a little first | 02:01 |
fungi | that makes your project a horribly unsuitable guinea pig for my completely unrelated needs | 02:01 |
* jhesketh__ is very unsure of what fungi is doing | 02:02 | |
fungi | hoping to confirm i didn't break tarball uploads | 02:02 |
jhesketh__ | ah right | 02:02 |
fungi | without going to the trouble of manually retriggering a job, which would be several seconds of my time wasted of course | 02:03 |
*** guohliu has joined #openstack-infra | 02:03 | |
*** jhesketh_ has quit IRC | 02:05 | |
*** melwitt1 has quit IRC | 02:07 | |
*** jhesketh_ has joined #openstack-infra | 02:10 | |
*** crank has quit IRC | 02:13 | |
*** crank has joined #openstack-infra | 02:14 | |
*** rongze has joined #openstack-infra | 02:14 | |
*** krotscheck1 has joined #openstack-infra | 02:18 | |
*** krotscheck1 has left #openstack-infra | 02:21 | |
*** ryanpetrello has quit IRC | 02:22 | |
*** zhiyan has quit IRC | 02:24 | |
*** ryanpetrello has joined #openstack-infra | 02:29 | |
*** krotscheck has joined #openstack-infra | 02:30 | |
*** crank has quit IRC | 02:31 | |
*** crank has joined #openstack-infra | 02:32 | |
openstackgerrit | Yuuichi Fujioka proposed a change to openstack-dev/hacking: checking for metaclass to be Python 3.x compatible https://review.openstack.org/56890 | 02:34 |
*** praneshp has quit IRC | 02:34 | |
*** praneshp has joined #openstack-infra | 02:34 | |
*** krotscheck has quit IRC | 02:40 | |
fungi | gah, upload failure... http://paste.openstack.org/show/55247/ | 02:40 |
fungi | do the jenkins masters need to have their known_hosts file entries for the old tarballs.o.o host key removed/replaced? | 02:41 |
fungi | clarkb: jeblair: mordred: ^ | 02:41 |
*** krotscheck_ has joined #openstack-infra | 02:43 | |
fungi | if so, it's not the ~jenkins/.ssh/known_hosts file on the masters at any rate | 02:43 |
fungi | and i confirmed that sudo -u jenkins ssh tarballs.openstack.org works and gets a shell on static | 02:44 |
anteaya | how does jenkins do auth? | 02:44 |
fungi | it has references to its ssh key on the filesystem (the one i just authenticated with) | 02:45 |
anteaya | hmm | 02:46 |
anteaya | the build step changed the result to unstable | 02:47 |
fungi | yep | 02:47 |
fungi | that's typical when it can't upload | 02:48 |
fungi | ohhhhhhhhhhhh | 02:48 |
fungi | the publisher's remote username was "hudson" on old-wiki | 02:48 |
fungi | fixing that to "jenkins" should solve it | 02:49 |
anteaya | ah | 02:51 |
*** freyes_ has quit IRC | 02:53 | |
fungi | updated on all 3 masters for completeness, and retriggered the job https://jenkins02.openstack.org/job/nova-branch-tarball/590/console | 02:54 |
*** vkozhukalov has joined #openstack-infra | 02:54 | |
anteaya | fungi I get finished unstable again | 02:57 |
fungi | yup, and i see why | 02:57 |
anteaya | do share, after you fix | 02:58 |
fungi | this time it's a permissions issue, because the files were owned by hudson when they were copied over. fixing | 02:58 |
*** praneshp has quit IRC | 02:58 | |
fungi | latest attempt running now... https://jenkins01.openstack.org/job/nova-branch-tarball/929/console | 02:59 |
fungi | that worked | 03:00 |
fungi | should be all set now | 03:00 |
fungi | confirmed that the timestamp for that branch tarball was updated at the new remote end | 03:01 |
* fungi switches to low-power mode for the rest of the evening | 03:01 | |
*** rongze has quit IRC | 03:02 | |
*** wenlock has joined #openstack-infra | 03:02 | |
*** mriedem has quit IRC | 03:04 | |
*** nati_uen_ has quit IRC | 03:04 | |
anteaya | yay | 03:05 |
*** rongze has joined #openstack-infra | 03:10 | |
*** alexpilotti has quit IRC | 03:11 | |
*** krotscheck_ has quit IRC | 03:13 | |
*** rnirmal has quit IRC | 03:15 | |
*** krotscheck has joined #openstack-infra | 03:15 | |
*** pcrews has quit IRC | 03:18 | |
*** gyee has quit IRC | 03:20 | |
*** krtaylor has quit IRC | 03:25 | |
*** ryanpetrello has quit IRC | 03:27 | |
jhesketh__ | If anybody has time to look at this simple fix I'd really appreciate it: https://review.openstack.org/#/c/62523/ | 03:27 |
*** UtahDave has quit IRC | 03:31 | |
*** AaronGr_Zzz is now known as AaronGr | 03:32 | |
fungi | jhesketh__: do you have a test gerrit you've tried that on? if not, i'll put it on my list of things to hopefully try out manually tomorrow | 03:33 |
*** slong has joined #openstack-infra | 03:34 | |
jhesketh__ | fungi: no, I was hoping a close review would be enough but I guess we really don't want to break gerrit | 03:34 |
*** krotscheck has quit IRC | 03:34 | |
fungi | well, i won't speak for others, but at least *i* personally break plenty enough already with out needing to tempt fate ;) | 03:36 |
jhesketh__ | heh, fair enough | 03:37 |
jhesketh__ | well I don't have a local jenkins/infra to test on, do you guys have a dev deployment? | 03:37 |
*** ArxCruz has quit IRC | 03:37 | |
fungi | of gerrit, yes and no. it's currently not usable for this because we're testing a gerrit version upgrade | 03:38 |
fungi | normally i'd say just tweak the change to only apply to review-dev first, but not for this week at any rate | 03:40 |
*** AaronGr is now known as AaronGr_Zzz | 03:40 | |
fungi | so i'll test rolling back our failed upgrade first and then give it a shot | 03:41 |
jhesketh__ | sure okay | 03:43 |
jhesketh__ | sounds good | 03:43 |
*** nicedice has quit IRC | 03:46 | |
*** nicedice has joined #openstack-infra | 03:46 | |
*** CaptTofu has quit IRC | 03:48 | |
*** CaptTofu has joined #openstack-infra | 03:48 | |
*** nicedice has quit IRC | 03:49 | |
*** nicedice has joined #openstack-infra | 03:50 | |
*** jhesketh__ has quit IRC | 03:50 | |
*** lyz has quit IRC | 03:50 | |
*** jhesketh__ has joined #openstack-infra | 03:50 | |
*** rcarrillocruz has joined #openstack-infra | 03:51 | |
*** harlowja is now known as harlowja_away | 03:51 | |
*** rcarrillocruz1 has joined #openstack-infra | 03:52 | |
*** ljjjustin has joined #openstack-infra | 03:54 | |
*** rcarrillocruz has quit IRC | 03:55 | |
*** rcarrillocruz1 has quit IRC | 03:56 | |
*** rcleere has joined #openstack-infra | 04:00 | |
*** CaptTofu has quit IRC | 04:10 | |
*** CaptTofu has joined #openstack-infra | 04:10 | |
*** ryanpetrello has joined #openstack-infra | 04:11 | |
*** niska has quit IRC | 04:14 | |
*** chandankumar has joined #openstack-infra | 04:15 | |
*** CaptTofu has quit IRC | 04:17 | |
*** julim has quit IRC | 04:18 | |
*** niska has joined #openstack-infra | 04:21 | |
*** slong has quit IRC | 04:24 | |
*** slong has joined #openstack-infra | 04:24 | |
*** talluri has joined #openstack-infra | 04:26 | |
*** CaptTofu has joined #openstack-infra | 04:33 | |
SpamapS | /win 13 | 04:35 |
*** guohliu has quit IRC | 04:36 | |
*** bingbu has joined #openstack-infra | 04:37 | |
openstackgerrit | Bill Maxwell proposed a change to openstack-infra/jenkins-job-builder: Implements: Archive publisher allow-empty setting. https://review.openstack.org/62806 | 04:39 |
*** Ryan_Lane has quit IRC | 04:47 | |
*** Ryan_Lane has joined #openstack-infra | 04:52 | |
*** Ryan_Lane has quit IRC | 04:56 | |
*** ryanpetrello has quit IRC | 04:58 | |
*** rongze has quit IRC | 04:59 | |
*** masayukig has joined #openstack-infra | 04:59 | |
*** talluri has quit IRC | 05:15 | |
*** Ryan_Lane has joined #openstack-infra | 05:18 | |
*** wenlock has quit IRC | 05:21 | |
*** Ryan_Lane has quit IRC | 05:23 | |
*** SergeyLukjanov has joined #openstack-infra | 05:24 | |
*** guohliu has joined #openstack-infra | 05:26 | |
*** CaptTofu has quit IRC | 05:29 | |
*** CaptTofu has joined #openstack-infra | 05:29 | |
*** rongze has joined #openstack-infra | 05:30 | |
*** praneshp has joined #openstack-infra | 05:30 | |
*** sdake_ has quit IRC | 05:31 | |
*** rongze has quit IRC | 05:35 | |
*** talluri has joined #openstack-infra | 05:38 | |
*** rcleere has quit IRC | 05:44 | |
*** crank has quit IRC | 05:45 | |
*** crank has joined #openstack-infra | 05:45 | |
*** davidhadas has joined #openstack-infra | 05:46 | |
*** slong has quit IRC | 05:50 | |
*** nati_ueno has joined #openstack-infra | 05:51 | |
*** slong has joined #openstack-infra | 05:54 | |
*** praneshp has quit IRC | 05:55 | |
*** SergeyLukjanov is now known as _SergeyLukjanov | 05:56 | |
*** _SergeyLukjanov has quit IRC | 05:56 | |
*** crank has quit IRC | 06:00 | |
*** crank has joined #openstack-infra | 06:01 | |
*** dkliban has quit IRC | 06:02 | |
*** fifieldt has joined #openstack-infra | 06:07 | |
*** rongze has joined #openstack-infra | 06:08 | |
*** zhiyan has joined #openstack-infra | 06:08 | |
*** SergeyLukjanov has joined #openstack-infra | 06:12 | |
*** vkozhukalov has quit IRC | 06:15 | |
*** vkozhukalov has joined #openstack-infra | 06:15 | |
*** senk has joined #openstack-infra | 06:17 | |
*** MarkAtwood has quit IRC | 06:18 | |
*** Ryan_Lane has joined #openstack-infra | 06:19 | |
*** Ryan_Lane has quit IRC | 06:24 | |
*** senk has quit IRC | 06:26 | |
zaro | ~. | 06:31 |
*** crank has quit IRC | 06:31 | |
*** crank has joined #openstack-infra | 06:32 | |
*** dkliban has joined #openstack-infra | 06:36 | |
*** sarob has joined #openstack-infra | 06:39 | |
*** dkehn has quit IRC | 06:40 | |
*** rwsu_ has quit IRC | 06:41 | |
*** dkehn has joined #openstack-infra | 06:42 | |
*** dstufft_ is now known as dstufft | 06:42 | |
*** praneshp has joined #openstack-infra | 06:49 | |
*** vkozhukalov has quit IRC | 06:52 | |
*** dims has quit IRC | 06:55 | |
*** davidhadas has quit IRC | 07:03 | |
*** yolanda has joined #openstack-infra | 07:08 | |
*** ljjjustin has quit IRC | 07:09 | |
*** jcooley_ has joined #openstack-infra | 07:10 | |
*** ljjjustin has joined #openstack-infra | 07:11 | |
*** pblaho has joined #openstack-infra | 07:12 | |
lifeless | btw https://bugs.launchpad.net/nova/+bug/1262075 might be the slowdown culprit? 196 seconds to rescue a server.. | 07:14 |
uvirtbot | Launchpad bug 1262075 in nova "tempest.api.compute.servers.test_server_rescue.ServerRescueTestXML.test_rescue_unrescue_instance[gate,smoke] failed on postgresql only" [Undecided,New] | 07:14 |
*** SergeyLukjanov has quit IRC | 07:14 | |
*** Ryan_Lane has joined #openstack-infra | 07:19 | |
*** dkliban has quit IRC | 07:19 | |
*** zhiyan has quit IRC | 07:20 | |
*** Ryan_Lane has quit IRC | 07:23 | |
*** yolanda has quit IRC | 07:24 | |
*** ljjjustin has quit IRC | 07:34 | |
*** sarob has quit IRC | 07:40 | |
*** sarob has joined #openstack-infra | 07:41 | |
*** zhiyan has joined #openstack-infra | 07:41 | |
*** dims_ has joined #openstack-infra | 07:43 | |
*** flaper87|afk is now known as flaper87 | 07:43 | |
*** jcooley_ has quit IRC | 07:45 | |
*** sarob has quit IRC | 07:46 | |
*** yolanda has joined #openstack-infra | 07:47 | |
*** dkliban has joined #openstack-infra | 07:56 | |
*** oubiwann has quit IRC | 07:58 | |
*** fifieldt has quit IRC | 08:02 | |
*** dkliban has quit IRC | 08:10 | |
*** sarob has joined #openstack-infra | 08:11 | |
*** jcoufal has joined #openstack-infra | 08:13 | |
*** davidhadas has joined #openstack-infra | 08:14 | |
*** flaper87 is now known as flaper87|afk | 08:14 | |
*** flaper87|afk is now known as flaper87 | 08:16 | |
*** sarob has quit IRC | 08:17 | |
*** jcoufal has quit IRC | 08:18 | |
*** Ryan_Lane has joined #openstack-infra | 08:19 | |
*** bingbu has quit IRC | 08:20 | |
*** Ryan_Lane has quit IRC | 08:20 | |
*** Ryan_Lane has joined #openstack-infra | 08:21 | |
*** SushilKM has joined #openstack-infra | 08:22 | |
*** shardy_afk is now known as shardy | 08:24 | |
*** Ryan_Lane has quit IRC | 08:25 | |
*** vkozhukalov has joined #openstack-infra | 08:31 | |
*** masayukig has quit IRC | 08:32 | |
*** SergeyLukjanov has joined #openstack-infra | 08:32 | |
*** boris-42 has joined #openstack-infra | 08:37 | |
*** mancdaz_away is now known as mancdaz | 08:38 | |
*** jcooley_ has joined #openstack-infra | 08:42 | |
*** thomasbiege has joined #openstack-infra | 08:43 | |
*** thomasbiege has quit IRC | 08:44 | |
*** jcooley_ has quit IRC | 08:48 | |
*** jcoufal has joined #openstack-infra | 08:52 | |
*** hashar has joined #openstack-infra | 08:53 | |
*** yassine has joined #openstack-infra | 08:54 | |
*** jcoufal_ has joined #openstack-infra | 08:56 | |
*** jcoufal has quit IRC | 08:58 | |
*** spredzy has quit IRC | 09:02 | |
*** mugsie has quit IRC | 09:04 | |
*** LinuxJedi_ has quit IRC | 09:09 | |
*** guohliu has quit IRC | 09:10 | |
*** katyafervent has quit IRC | 09:10 | |
*** ianw has quit IRC | 09:10 | |
*** Hefeweizen has quit IRC | 09:10 | |
*** talluri has quit IRC | 09:10 | |
*** markmcclain has quit IRC | 09:10 | |
*** salv-orlando has quit IRC | 09:10 | |
*** simonmcc has quit IRC | 09:10 | |
*** openstackgerrit has quit IRC | 09:11 | |
*** med_ has quit IRC | 09:11 | |
*** samalba has quit IRC | 09:11 | |
*** jeblair has quit IRC | 09:11 | |
*** jcooley has quit IRC | 09:11 | |
*** dragondm has quit IRC | 09:11 | |
*** vkozhukalov has quit IRC | 09:11 | |
*** yolanda has quit IRC | 09:11 | |
*** sdake has quit IRC | 09:11 | |
*** esker has quit IRC | 09:11 | |
*** bknudson has quit IRC | 09:11 | |
*** briancline has quit IRC | 09:11 | |
*** freyes has quit IRC | 09:11 | |
*** mgagne has quit IRC | 09:11 | |
*** vishy has quit IRC | 09:11 | |
*** zigo has quit IRC | 09:11 | |
*** Adri2000 has quit IRC | 09:11 | |
*** cyeoh has quit IRC | 09:11 | |
*** persia has quit IRC | 09:11 | |
*** morganfainberg has quit IRC | 09:11 | |
*** pblaho has quit IRC | 09:11 | |
*** nati_ueno has quit IRC | 09:11 | |
*** blamar has quit IRC | 09:11 | |
*** branen has quit IRC | 09:11 | |
*** vipul has quit IRC | 09:11 | |
*** HenryG has quit IRC | 09:11 | |
*** akscram has quit IRC | 09:11 | |
*** ilyashakhat has quit IRC | 09:11 | |
*** agordeev has quit IRC | 09:11 | |
*** dkranz has quit IRC | 09:11 | |
*** dims_ has quit IRC | 09:11 | |
*** praneshp has quit IRC | 09:11 | |
*** dkehn has quit IRC | 09:11 | |
*** jhesketh__ has quit IRC | 09:11 | |
*** yaguang has quit IRC | 09:11 | |
*** rockstar has quit IRC | 09:11 | |
*** portante has quit IRC | 09:11 | |
*** jpeeler has quit IRC | 09:11 | |
*** thingee has quit IRC | 09:11 | |
*** basic` has quit IRC | 09:11 | |
*** pleia2 has quit IRC | 09:11 | |
*** SpamapS has quit IRC | 09:11 | |
*** odyi has quit IRC | 09:11 | |
*** hub_cap has quit IRC | 09:11 | |
*** koolhead17 has quit IRC | 09:11 | |
*** hemanth_ has quit IRC | 09:11 | |
*** kiall has quit IRC | 09:11 | |
*** BobBall has quit IRC | 09:11 | |
*** rahmu has quit IRC | 09:11 | |
*** metabro_ has quit IRC | 09:11 | |
*** chmouel has quit IRC | 09:11 | |
*** Daviey has quit IRC | 09:11 | |
*** wayneeseguin has quit IRC | 09:11 | |
*** SlickNik has quit IRC | 09:11 | |
*** Mithrandir has quit IRC | 09:11 | |
*** antonym has quit IRC | 09:11 | |
*** sdague has quit IRC | 09:11 | |
*** EntropyWorks has quit IRC | 09:11 | |
*** jgriffith has quit IRC | 09:11 | |
*** mika has quit IRC | 09:11 | |
*** rainya has quit IRC | 09:11 | |
*** esmute has quit IRC | 09:11 | |
*** adam_g has quit IRC | 09:11 | |
*** stevebaker has quit IRC | 09:11 | |
*** ttx has quit IRC | 09:11 | |
*** kpepple has quit IRC | 09:11 | |
*** jcoufal_ has quit IRC | 09:11 | |
*** niska has quit IRC | 09:11 | |
*** nosnos has quit IRC | 09:11 | |
*** danger_f` has quit IRC | 09:11 | |
*** jog0 has quit IRC | 09:11 | |
*** harlowja_away has quit IRC | 09:11 | |
*** lifeless has quit IRC | 09:11 | |
*** dstufft has quit IRC | 09:11 | |
*** kashyap has quit IRC | 09:11 | |
*** tian has quit IRC | 09:11 | |
*** jswarren has quit IRC | 09:11 | |
*** miqui has quit IRC | 09:11 | |
*** jerryz has quit IRC | 09:11 | |
*** ywu has quit IRC | 09:11 | |
*** hashar has quit IRC | 09:11 | |
*** boris-42 has quit IRC | 09:11 | |
*** crank has quit IRC | 09:11 | |
*** sandywalsh_ has quit IRC | 09:11 | |
*** shardy has quit IRC | 09:11 | |
*** sirushti has quit IRC | 09:11 | |
*** primeministerp has quit IRC | 09:11 | |
*** devananda has quit IRC | 09:11 | |
*** gema has quit IRC | 09:11 | |
*** avishayb has quit IRC | 09:11 | |
*** davidhadas has quit IRC | 09:11 | |
*** chandankumar has quit IRC | 09:11 | |
*** mkerrin has quit IRC | 09:11 | |
*** StevenK has quit IRC | 09:11 | |
*** pabelanger has quit IRC | 09:11 | |
*** jd__ has quit IRC | 09:11 | |
*** ekarlso has quit IRC | 09:11 | |
*** Grishkin has quit IRC | 09:11 | |
*** milki has quit IRC | 09:11 | |
*** SergeyLukjanov has quit IRC | 09:11 | |
*** zhiyan has quit IRC | 09:11 | |
*** rongze has quit IRC | 09:11 | |
*** yamahata has quit IRC | 09:11 | |
*** gaelL has quit IRC | 09:11 | |
*** juice has quit IRC | 09:11 | |
*** mattymo has quit IRC | 09:11 | |
*** jheiss has quit IRC | 09:11 | |
*** vladan has quit IRC | 09:11 | |
*** uvirtbot has quit IRC | 09:11 | |
*** jesusaurus has quit IRC | 09:11 | |
*** bogdando has quit IRC | 09:11 | |
*** rpodolyaka has quit IRC | 09:11 | |
*** yassine has quit IRC | 09:11 | |
*** jhesketh_ has quit IRC | 09:11 | |
*** Shrews has quit IRC | 09:11 | |
*** jaypipes has quit IRC | 09:11 | |
*** mordred has quit IRC | 09:11 | |
*** dansmith has quit IRC | 09:11 | |
*** spiffxp_ has quit IRC | 09:11 | |
*** Loquacities has quit IRC | 09:11 | |
*** pvo has quit IRC | 09:11 | |
*** Guest33069 has quit IRC | 09:11 | |
*** AaronGr_Zzz has quit IRC | 09:11 | |
*** sileht has quit IRC | 09:11 | |
*** mancdaz has quit IRC | 09:11 | |
*** jlk has quit IRC | 09:11 | |
*** EmilienM has quit IRC | 09:11 | |
*** russellb has quit IRC | 09:11 | |
*** yongli has quit IRC | 09:11 | |
*** dhellmann has quit IRC | 09:11 | |
*** torgomatic has quit IRC | 09:11 | |
*** davidlenwell has quit IRC | 09:11 | |
*** iccha-dinner has quit IRC | 09:11 | |
*** hughsaunders has quit IRC | 09:11 | |
*** maurosr has quit IRC | 09:11 | |
*** saper has quit IRC | 09:11 | |
*** Protux has quit IRC | 09:11 | |
*** olaph has quit IRC | 09:11 | |
*** zaro has quit IRC | 09:11 | |
*** mindjiver has quit IRC | 09:11 | |
*** david-lyle has quit IRC | 09:11 | |
*** alaski has quit IRC | 09:11 | |
*** _cyril_ has quit IRC | 09:11 | |
*** guitarzan has quit IRC | 09:11 | |
*** GheRivero has quit IRC | 09:11 | |
*** hdd has quit IRC | 09:11 | |
*** clarkb has quit IRC | 09:11 | |
*** Ng has quit IRC | 09:11 | |
*** dachary has quit IRC | 09:11 | |
*** mikal has quit IRC | 09:11 | |
*** nikhil__ has quit IRC | 09:11 | |
*** westmau5 has quit IRC | 09:11 | |
*** jeremyb has quit IRC | 09:11 | |
*** greghaynes has quit IRC | 09:11 | |
*** Alex_Gaynor has quit IRC | 09:11 | |
*** fbo_away has quit IRC | 09:11 | |
*** notmyname has quit IRC | 09:11 | |
*** gfa has quit IRC | 09:11 | |
*** ewindisch has quit IRC | 09:11 | |
*** ogelbukh has quit IRC | 09:11 | |
*** adarazs has quit IRC | 09:11 | |
*** annegentle_ has quit IRC | 09:11 | |
*** enikanorov has quit IRC | 09:11 | |
*** mkoderer_ has quit IRC | 09:11 | |
*** tsufiev_ has quit IRC | 09:11 | |
*** belliott has quit IRC | 09:11 | |
*** nijaba has quit IRC | 09:11 | |
*** SushilKM has quit IRC | 09:11 | |
*** slong has quit IRC | 09:11 | |
*** CaptTofu has quit IRC | 09:11 | |
*** nicedice has quit IRC | 09:11 | |
*** zul has quit IRC | 09:11 | |
*** bnemec has quit IRC | 09:11 | |
*** ChanServ has quit IRC | 09:11 | |
*** Hunner has quit IRC | 09:11 | |
*** andreaf has quit IRC | 09:11 | |
*** adalbas has quit IRC | 09:11 | |
*** luisg has quit IRC | 09:11 | |
*** mtreinish has quit IRC | 09:11 | |
*** yamahata__ has quit IRC | 09:11 | |
*** marun has quit IRC | 09:11 | |
*** michchap has quit IRC | 09:11 | |
*** fungi has quit IRC | 09:11 | |
*** sbadia has quit IRC | 09:11 | |
*** anteaya has quit IRC | 09:11 | |
*** LinuxJedi has joined #openstack-infra | 09:12 | |
*** thomasbiege has joined #openstack-infra | 09:12 | |
*** afazekas has joined #openstack-infra | 09:12 | |
*** jpich has joined #openstack-infra | 09:12 | |
*** mugsie_ has joined #openstack-infra | 09:12 | |
*** spredzy_ has joined #openstack-infra | 09:12 | |
*** jcoufal has joined #openstack-infra | 09:12 | |
*** yassine has joined #openstack-infra | 09:12 | |
*** hashar has joined #openstack-infra | 09:12 | |
*** boris-42 has joined #openstack-infra | 09:12 | |
*** SergeyLukjanov has joined #openstack-infra | 09:12 | |
*** vkozhukalov has joined #openstack-infra | 09:12 | |
*** SushilKM has joined #openstack-infra | 09:12 | |
*** davidhadas has joined #openstack-infra | 09:12 | |
*** yolanda has joined #openstack-infra | 09:12 | |
*** dims_ has joined #openstack-infra | 09:12 | |
*** zhiyan has joined #openstack-infra | 09:12 | |
*** pblaho has joined #openstack-infra | 09:12 | |
*** praneshp has joined #openstack-infra | 09:12 | |
*** dkehn has joined #openstack-infra | 09:12 | |
*** crank has joined #openstack-infra | 09:12 | |
*** rongze has joined #openstack-infra | 09:12 | |
*** slong has joined #openstack-infra | 09:12 | |
*** nati_ueno has joined #openstack-infra | 09:12 | |
*** talluri has joined #openstack-infra | 09:12 | |
*** CaptTofu has joined #openstack-infra | 09:12 | |
*** guohliu has joined #openstack-infra | 09:12 | |
*** niska has joined #openstack-infra | 09:12 | |
*** chandankumar has joined #openstack-infra | 09:12 | |
*** jhesketh__ has joined #openstack-infra | 09:12 | |
*** nicedice has joined #openstack-infra | 09:12 | |
*** jhesketh_ has joined #openstack-infra | 09:12 | |
*** sdake has joined #openstack-infra | 09:12 | |
*** yaguang has joined #openstack-infra | 09:12 | |
*** nosnos has joined #openstack-infra | 09:12 | |
*** blamar has joined #openstack-infra | 09:12 | |
*** esker has joined #openstack-infra | 09:12 | |
*** yamahata has joined #openstack-infra | 09:12 | |
*** markmcclain has joined #openstack-infra | 09:12 | |
*** branen has joined #openstack-infra | 09:12 | |
*** bknudson has joined #openstack-infra | 09:12 | |
*** briancline has joined #openstack-infra | 09:12 | |
*** sandywalsh_ has joined #openstack-infra | 09:12 | |
*** Hunner has joined #openstack-infra | 09:12 | |
*** miqui has joined #openstack-infra | 09:12 | |
*** salv-orlando has joined #openstack-infra | 09:12 | |
*** vipul has joined #openstack-infra | 09:12 | |
*** freyes has joined #openstack-infra | 09:12 | |
*** simonmcc has joined #openstack-infra | 09:12 | |
*** andreaf has joined #openstack-infra | 09:12 | |
*** jswarren has joined #openstack-infra | 09:12 | |
*** katyafervent has joined #openstack-infra | 09:12 | |
*** adalbas has joined #openstack-infra | 09:12 | |
*** HenryG has joined #openstack-infra | 09:12 | |
*** luisg has joined #openstack-infra | 09:12 | |
*** rpodolyaka has joined #openstack-infra | 09:12 | |
*** _cyril_ has joined #openstack-infra | 09:12 | |
*** mtreinish has joined #openstack-infra | 09:12 | |
*** yamahata__ has joined #openstack-infra | 09:12 | |
*** akscram has joined #openstack-infra | 09:12 | |
*** ilyashakhat has joined #openstack-infra | 09:12 | |
*** marun has joined #openstack-infra | 09:12 | |
*** zul has joined #openstack-infra | 09:12 | |
*** agordeev has joined #openstack-infra | 09:12 | |
*** avishayb has joined #openstack-infra | 09:12 | |
*** michchap has joined #openstack-infra | 09:12 | |
*** greghaynes has joined #openstack-infra | 09:12 | |
*** dkranz has joined #openstack-infra | 09:12 | |
*** mgagne has joined #openstack-infra | 09:12 | |
*** ianw has joined #openstack-infra | 09:12 | |
*** openstackgerrit has joined #openstack-infra | 09:12 | |
*** bnemec has joined #openstack-infra | 09:12 | |
*** bogdando has joined #openstack-infra | 09:12 | |
*** Hefeweizen has joined #openstack-infra | 09:12 | |
*** jerryz has joined #openstack-infra | 09:12 | |
*** fungi has joined #openstack-infra | 09:12 | |
*** shardy has joined #openstack-infra | 09:12 | |
*** Shrews has joined #openstack-infra | 09:12 | |
*** rockstar has joined #openstack-infra | 09:12 | |
*** jaypipes has joined #openstack-infra | 09:12 | |
*** portante has joined #openstack-infra | 09:12 | |
*** mordred has joined #openstack-infra | 09:12 | |
*** dansmith has joined #openstack-infra | 09:12 | |
*** dragondm has joined #openstack-infra | 09:12 | |
*** BobBall has joined #openstack-infra | 09:12 | |
*** jcooley has joined #openstack-infra | 09:12 | |
*** kiall has joined #openstack-infra | 09:12 | |
*** jeblair has joined #openstack-infra | 09:12 | |
*** samalba has joined #openstack-infra | 09:12 | |
*** koolhead17 has joined #openstack-infra | 09:12 | |
*** med_ has joined #openstack-infra | 09:12 | |
*** kpepple has joined #openstack-infra | 09:12 | |
*** ttx has joined #openstack-infra | 09:12 | |
*** stevebaker has joined #openstack-infra | 09:12 | |
*** adam_g has joined #openstack-infra | 09:12 | |
*** hub_cap has joined #openstack-infra | 09:12 | |
*** esmute has joined #openstack-infra | 09:12 | |
*** odyi has joined #openstack-infra | 09:12 | |
*** mika has joined #openstack-infra | 09:12 | |
*** jgriffith has joined #openstack-infra | 09:12 | |
*** EntropyWorks has joined #openstack-infra | 09:12 | |
*** SpamapS has joined #openstack-infra | 09:12 | |
*** pleia2 has joined #openstack-infra | 09:12 | |
*** basic` has joined #openstack-infra | 09:12 | |
*** thingee has joined #openstack-infra | 09:12 | |
*** hemanth_ has joined #openstack-infra | 09:12 | |
*** jpeeler has joined #openstack-infra | 09:12 | |
*** sdague has joined #openstack-infra | 09:12 | |
*** rainya has joined #openstack-infra | 09:12 | |
*** antonym has joined #openstack-infra | 09:12 | |
*** Mithrandir has joined #openstack-infra | 09:12 | |
*** vishy has joined #openstack-infra | 09:12 | |
*** zigo has joined #openstack-infra | 09:12 | |
*** Adri2000 has joined #openstack-infra | 09:12 | |
*** cyeoh has joined #openstack-infra | 09:12 | |
*** persia has joined #openstack-infra | 09:12 | |
*** morganfainberg has joined #openstack-infra | 09:12 | |
*** SlickNik has joined #openstack-infra | 09:12 | |
*** wayneeseguin has joined #openstack-infra | 09:12 | |
*** chmouel has joined #openstack-infra | 09:12 | |
*** metabro_ has joined #openstack-infra | 09:12 | |
*** rahmu has joined #openstack-infra | 09:12 | |
*** Daviey has joined #openstack-infra | 09:12 | |
*** spiffxp_ has joined #openstack-infra | 09:12 | |
*** mkerrin has joined #openstack-infra | 09:12 | |
*** yongli has joined #openstack-infra | 09:12 | |
*** dhellmann has joined #openstack-infra | 09:12 | |
*** torgomatic has joined #openstack-infra | 09:12 | |
*** danger_f` has joined #openstack-infra | 09:12 | |
*** Alex_Gaynor has joined #openstack-infra | 09:12 | |
*** annegentle_ has joined #openstack-infra | 09:12 | |
*** sirushti has joined #openstack-infra | 09:12 | |
*** davidlenwell has joined #openstack-infra | 09:12 | |
*** primeministerp has joined #openstack-infra | 09:12 | |
*** fbo_away has joined #openstack-infra | 09:12 | |
*** notmyname has joined #openstack-infra | 09:12 | |
*** devananda has joined #openstack-infra | 09:12 | |
*** gfa has joined #openstack-infra | 09:12 | |
*** jog0 has joined #openstack-infra | 09:12 | |
*** ewindisch has joined #openstack-infra | 09:12 | |
*** ogelbukh has joined #openstack-infra | 09:12 | |
*** adarazs has joined #openstack-infra | 09:12 | |
*** harlowja_away has joined #openstack-infra | 09:12 | |
*** enikanorov has joined #openstack-infra | 09:12 | |
*** lifeless has joined #openstack-infra | 09:12 | |
*** mkoderer_ has joined #openstack-infra | 09:12 | |
*** belliott has joined #openstack-infra | 09:12 | |
*** tsufiev_ has joined #openstack-infra | 09:12 | |
*** dstufft has joined #openstack-infra | 09:12 | |
*** kashyap has joined #openstack-infra | 09:12 | |
*** gaelL has joined #openstack-infra | 09:12 | |
*** tian has joined #openstack-infra | 09:12 | |
*** juice has joined #openstack-infra | 09:12 | |
*** guitarzan has joined #openstack-infra | 09:12 | |
*** GheRivero has joined #openstack-infra | 09:12 | |
*** iccha-dinner has joined #openstack-infra | 09:12 | |
*** StevenK has joined #openstack-infra | 09:12 | |
*** hughsaunders has joined #openstack-infra | 09:12 | |
*** maurosr has joined #openstack-infra | 09:12 | |
*** pabelanger has joined #openstack-infra | 09:12 | |
*** jd__ has joined #openstack-infra | 09:12 | |
*** saper has joined #openstack-infra | 09:12 | |
*** gema has joined #openstack-infra | 09:12 | |
*** ekarlso has joined #openstack-infra | 09:12 | |
*** ywu has joined #openstack-infra | 09:12 | |
*** Loquacities has joined #openstack-infra | 09:12 | |
*** Protux has joined #openstack-infra | 09:12 | |
*** olaph has joined #openstack-infra | 09:12 | |
*** Grishkin has joined #openstack-infra | 09:12 | |
*** pvo has joined #openstack-infra | 09:12 | |
*** Guest33069 has joined #openstack-infra | 09:12 | |
*** hdd has joined #openstack-infra | 09:12 | |
*** AaronGr_Zzz has joined #openstack-infra | 09:12 | |
*** mattymo has joined #openstack-infra | 09:12 | |
*** clarkb has joined #openstack-infra | 09:12 | |
*** zaro has joined #openstack-infra | 09:12 | |
*** sileht has joined #openstack-infra | 09:12 | |
*** mancdaz has joined #openstack-infra | 09:12 | |
*** milki has joined #openstack-infra | 09:12 | |
*** mindjiver has joined #openstack-infra | 09:12 | |
*** sbadia has joined #openstack-infra | 09:12 | |
*** jheiss has joined #openstack-infra | 09:12 | |
*** Ng has joined #openstack-infra | 09:12 | |
*** david-lyle has joined #openstack-infra | 09:12 | |
*** jlk has joined #openstack-infra | 09:12 | |
*** vladan has joined #openstack-infra | 09:12 | |
*** dachary has joined #openstack-infra | 09:12 | |
*** uvirtbot has joined #openstack-infra | 09:12 | |
*** EmilienM has joined #openstack-infra | 09:12 | |
*** russellb has joined #openstack-infra | 09:12 | |
*** jesusaurus has joined #openstack-infra | 09:12 | |
*** mikal has joined #openstack-infra | 09:12 | |
*** alaski has joined #openstack-infra | 09:12 | |
*** nikhil__ has joined #openstack-infra | 09:12 | |
*** anteaya has joined #openstack-infra | 09:12 | |
*** nijaba has joined #openstack-infra | 09:12 | |
*** westmau5 has joined #openstack-infra | 09:12 | |
*** jeremyb has joined #openstack-infra | 09:12 | |
*** ChanServ has joined #openstack-infra | 09:12 | |
*** hobana.freenode.net sets mode: +o ChanServ | 09:12 | |
*** flaper87 has quit IRC | 09:13 | |
*** flaper87 has joined #openstack-infra | 09:13 | |
*** sarob has joined #openstack-infra | 09:13 | |
*** crank has quit IRC | 09:15 | |
*** guohliu has quit IRC | 09:16 | |
*** katyafervent has quit IRC | 09:16 | |
*** ianw has quit IRC | 09:16 | |
*** Hefeweizen has quit IRC | 09:16 | |
*** talluri has quit IRC | 09:16 | |
*** markmcclain has quit IRC | 09:16 | |
*** salv-orlando has quit IRC | 09:16 | |
*** simonmcc has quit IRC | 09:16 | |
*** openstackgerrit has quit IRC | 09:16 | |
*** med_ has quit IRC | 09:16 | |
*** samalba has quit IRC | 09:16 | |
*** jeblair has quit IRC | 09:16 | |
*** jcooley has quit IRC | 09:16 | |
*** dragondm has quit IRC | 09:16 | |
*** sarob has quit IRC | 09:16 | |
*** vkozhukalov has quit IRC | 09:16 | |
*** yolanda has quit IRC | 09:16 | |
*** sdake has quit IRC | 09:16 | |
*** esker has quit IRC | 09:16 | |
*** bknudson has quit IRC | 09:16 | |
*** briancline has quit IRC | 09:16 | |
*** freyes has quit IRC | 09:16 | |
*** mgagne has quit IRC | 09:16 | |
*** vishy has quit IRC | 09:16 | |
*** zigo has quit IRC | 09:16 | |
*** Adri2000 has quit IRC | 09:16 | |
*** cyeoh has quit IRC | 09:16 | |
*** persia has quit IRC | 09:16 | |
*** morganfainberg has quit IRC | 09:16 | |
*** spredzy_ has quit IRC | 09:16 | |
*** pblaho has quit IRC | 09:16 | |
*** nati_ueno has quit IRC | 09:16 | |
*** blamar has quit IRC | 09:16 | |
*** branen has quit IRC | 09:16 | |
*** vipul has quit IRC | 09:16 | |
*** HenryG has quit IRC | 09:16 | |
*** akscram has quit IRC | 09:16 | |
*** ilyashakhat has quit IRC | 09:16 | |
*** agordeev has quit IRC | 09:16 | |
*** dkranz has quit IRC | 09:16 | |
*** dims_ has quit IRC | 09:16 | |
*** praneshp has quit IRC | 09:16 | |
*** dkehn has quit IRC | 09:16 | |
*** jhesketh__ has quit IRC | 09:16 | |
*** yaguang has quit IRC | 09:16 | |
*** rockstar has quit IRC | 09:16 | |
*** portante has quit IRC | 09:16 | |
*** jpeeler has quit IRC | 09:16 | |
*** thingee has quit IRC | 09:16 | |
*** basic` has quit IRC | 09:16 | |
*** pleia2 has quit IRC | 09:16 | |
*** SpamapS has quit IRC | 09:16 | |
*** odyi has quit IRC | 09:16 | |
*** hub_cap has quit IRC | 09:16 | |
*** koolhead17 has quit IRC | 09:16 | |
*** hemanth_ has quit IRC | 09:16 | |
*** kiall has quit IRC | 09:16 | |
*** BobBall has quit IRC | 09:16 | |
*** rahmu has quit IRC | 09:16 | |
*** metabro_ has quit IRC | 09:16 | |
*** chmouel has quit IRC | 09:16 | |
*** Daviey has quit IRC | 09:16 | |
*** wayneeseguin has quit IRC | 09:16 | |
*** SlickNik has quit IRC | 09:16 | |
*** Mithrandir has quit IRC | 09:16 | |
*** antonym has quit IRC | 09:16 | |
*** sdague has quit IRC | 09:16 | |
*** EntropyWorks has quit IRC | 09:16 | |
*** jgriffith has quit IRC | 09:16 | |
*** mika has quit IRC | 09:16 | |
*** rainya has quit IRC | 09:16 | |
*** esmute has quit IRC | 09:16 | |
*** adam_g has quit IRC | 09:16 | |
*** stevebaker has quit IRC | 09:16 | |
*** ttx has quit IRC | 09:16 | |
*** kpepple has quit IRC | 09:16 | |
*** LinuxJedi has quit IRC | 09:16 | |
*** afazekas has quit IRC | 09:16 | |
*** jcoufal has quit IRC | 09:16 | |
*** niska has quit IRC | 09:16 | |
*** nosnos has quit IRC | 09:16 | |
*** danger_f` has quit IRC | 09:16 | |
*** jog0 has quit IRC | 09:16 | |
*** harlowja_away has quit IRC | 09:16 | |
*** lifeless has quit IRC | 09:16 | |
*** dstufft has quit IRC | 09:16 | |
*** kashyap has quit IRC | 09:16 | |
*** tian has quit IRC | 09:16 | |
*** jswarren has quit IRC | 09:16 | |
*** miqui has quit IRC | 09:16 | |
*** jerryz has quit IRC | 09:16 | |
*** ywu has quit IRC | 09:16 | |
*** hashar has quit IRC | 09:16 | |
*** boris-42 has quit IRC | 09:16 | |
*** sandywalsh_ has quit IRC | 09:16 | |
*** shardy has quit IRC | 09:16 | |
*** sirushti has quit IRC | 09:16 | |
*** primeministerp has quit IRC | 09:16 | |
*** devananda has quit IRC | 09:16 | |
*** gema has quit IRC | 09:16 | |
*** avishayb has quit IRC | 09:16 | |
*** davidhadas has quit IRC | 09:16 | |
*** chandankumar has quit IRC | 09:16 | |
*** mkerrin has quit IRC | 09:16 | |
*** StevenK has quit IRC | 09:16 | |
*** pabelanger has quit IRC | 09:16 | |
*** jd__ has quit IRC | 09:16 | |
*** ekarlso has quit IRC | 09:16 | |
*** Grishkin has quit IRC | 09:16 | |
*** milki has quit IRC | 09:16 | |
*** jpich has quit IRC | 09:16 | |
*** mugsie_ has quit IRC | 09:16 | |
*** SergeyLukjanov has quit IRC | 09:16 | |
*** zhiyan has quit IRC | 09:16 | |
*** rongze has quit IRC | 09:16 | |
*** yamahata has quit IRC | 09:16 | |
*** gaelL has quit IRC | 09:16 | |
*** juice has quit IRC | 09:16 | |
*** mattymo has quit IRC | 09:16 | |
*** jheiss has quit IRC | 09:16 | |
*** vladan has quit IRC | 09:16 | |
*** uvirtbot has quit IRC | 09:16 | |
*** jesusaurus has quit IRC | 09:16 | |
*** bogdando has quit IRC | 09:16 | |
*** rpodolyaka has quit IRC | 09:16 | |
*** thomasbiege has quit IRC | 09:16 | |
*** yassine has quit IRC | 09:16 | |
*** jhesketh_ has quit IRC | 09:16 | |
*** Shrews has quit IRC | 09:16 | |
*** jaypipes has quit IRC | 09:16 | |
*** mordred has quit IRC | 09:16 | |
*** dansmith has quit IRC | 09:16 | |
*** spiffxp_ has quit IRC | 09:16 | |
*** Loquacities has quit IRC | 09:16 | |
*** pvo has quit IRC | 09:16 | |
*** Guest33069 has quit IRC | 09:16 | |
*** AaronGr_Zzz has quit IRC | 09:16 | |
*** sileht has quit IRC | 09:16 | |
*** mancdaz has quit IRC | 09:16 | |
*** jlk has quit IRC | 09:16 | |
*** EmilienM has quit IRC | 09:16 | |
*** russellb has quit IRC | 09:16 | |
*** yongli has quit IRC | 09:16 | |
*** dhellmann has quit IRC | 09:16 | |
*** torgomatic has quit IRC | 09:16 | |
*** davidlenwell has quit IRC | 09:16 | |
*** iccha-dinner has quit IRC | 09:16 | |
*** hughsaunders has quit IRC | 09:16 | |
*** maurosr has quit IRC | 09:16 | |
*** saper has quit IRC | 09:16 | |
*** Protux has quit IRC | 09:16 | |
*** olaph has quit IRC | 09:16 | |
*** zaro has quit IRC | 09:16 | |
*** mindjiver has quit IRC | 09:16 | |
*** david-lyle has quit IRC | 09:16 | |
*** alaski has quit IRC | 09:16 | |
*** _cyril_ has quit IRC | 09:16 | |
*** guitarzan has quit IRC | 09:16 | |
*** GheRivero has quit IRC | 09:16 | |
*** hdd has quit IRC | 09:16 | |
*** clarkb has quit IRC | 09:16 | |
*** Ng has quit IRC | 09:16 | |
*** dachary has quit IRC | 09:16 | |
*** mikal has quit IRC | 09:16 | |
*** nikhil__ has quit IRC | 09:16 | |
*** westmau5 has quit IRC | 09:16 | |
*** jeremyb has quit IRC | 09:16 | |
*** greghaynes has quit IRC | 09:16 | |
*** Alex_Gaynor has quit IRC | 09:16 | |
*** fbo_away has quit IRC | 09:16 | |
*** notmyname has quit IRC | 09:16 | |
*** gfa has quit IRC | 09:16 | |
*** ewindisch has quit IRC | 09:16 | |
*** ogelbukh has quit IRC | 09:16 | |
*** adarazs has quit IRC | 09:16 | |
*** annegentle_ has quit IRC | 09:16 | |
*** enikanorov has quit IRC | 09:16 | |
*** mkoderer_ has quit IRC | 09:16 | |
*** tsufiev_ has quit IRC | 09:16 | |
*** belliott has quit IRC | 09:16 | |
*** nijaba has quit IRC | 09:16 | |
*** SushilKM has quit IRC | 09:16 | |
*** slong has quit IRC | 09:16 | |
*** CaptTofu has quit IRC | 09:16 | |
*** nicedice has quit IRC | 09:16 | |
*** zul has quit IRC | 09:16 | |
*** bnemec has quit IRC | 09:16 | |
*** ChanServ has quit IRC | 09:17 | |
*** Hunner has quit IRC | 09:17 | |
*** andreaf has quit IRC | 09:17 | |
*** adalbas has quit IRC | 09:17 | |
*** luisg has quit IRC | 09:17 | |
*** mtreinish has quit IRC | 09:17 | |
*** yamahata__ has quit IRC | 09:17 | |
*** marun has quit IRC | 09:17 | |
*** michchap has quit IRC | 09:17 | |
*** fungi has quit IRC | 09:17 | |
*** sbadia has quit IRC | 09:17 | |
*** anteaya has quit IRC | 09:17 | |
*** Ryan_Lane has joined #openstack-infra | 09:22 | |
*** derekh has joined #openstack-infra | 09:22 | |
*** LinuxJedi has joined #openstack-infra | 09:22 | |
*** thomasbiege has joined #openstack-infra | 09:22 | |
*** afazekas has joined #openstack-infra | 09:22 | |
*** jpich has joined #openstack-infra | 09:22 | |
*** mugsie_ has joined #openstack-infra | 09:22 | |
*** spredzy_ has joined #openstack-infra | 09:22 | |
*** jcoufal has joined #openstack-infra | 09:22 | |
*** yassine has joined #openstack-infra | 09:22 | |
*** hashar has joined #openstack-infra | 09:22 | |
*** boris-42 has joined #openstack-infra | 09:22 | |
*** SergeyLukjanov has joined #openstack-infra | 09:22 | |
*** vkozhukalov has joined #openstack-infra | 09:22 | |
*** SushilKM has joined #openstack-infra | 09:22 | |
*** davidhadas has joined #openstack-infra | 09:22 | |
*** yolanda has joined #openstack-infra | 09:22 | |
*** dims_ has joined #openstack-infra | 09:22 | |
*** zhiyan has joined #openstack-infra | 09:22 | |
*** pblaho has joined #openstack-infra | 09:22 | |
*** praneshp has joined #openstack-infra | 09:22 | |
*** dkehn has joined #openstack-infra | 09:22 | |
*** rongze has joined #openstack-infra | 09:22 | |
*** slong has joined #openstack-infra | 09:22 | |
*** nati_ueno has joined #openstack-infra | 09:22 | |
*** talluri has joined #openstack-infra | 09:22 | |
*** CaptTofu has joined #openstack-infra | 09:22 | |
*** guohliu has joined #openstack-infra | 09:22 | |
*** niska has joined #openstack-infra | 09:22 | |
*** chandankumar has joined #openstack-infra | 09:22 | |
*** nicedice has joined #openstack-infra | 09:22 | |
*** sdake has joined #openstack-infra | 09:22 | |
*** yaguang has joined #openstack-infra | 09:22 | |
*** nosnos has joined #openstack-infra | 09:22 | |
*** blamar has joined #openstack-infra | 09:22 | |
*** esker has joined #openstack-infra | 09:22 | |
*** yamahata has joined #openstack-infra | 09:22 | |
*** markmcclain has joined #openstack-infra | 09:22 | |
*** branen has joined #openstack-infra | 09:22 | |
*** bknudson has joined #openstack-infra | 09:22 | |
*** briancline has joined #openstack-infra | 09:22 | |
*** sandywalsh_ has joined #openstack-infra | 09:22 | |
*** Hunner has joined #openstack-infra | 09:22 | |
*** miqui has joined #openstack-infra | 09:22 | |
*** salv-orlando has joined #openstack-infra | 09:22 | |
*** vipul has joined #openstack-infra | 09:22 | |
*** freyes has joined #openstack-infra | 09:22 | |
*** simonmcc has joined #openstack-infra | 09:22 | |
*** andreaf has joined #openstack-infra | 09:22 | |
*** jswarren has joined #openstack-infra | 09:22 | |
*** katyafervent has joined #openstack-infra | 09:22 | |
*** adalbas has joined #openstack-infra | 09:22 | |
*** HenryG has joined #openstack-infra | 09:22 | |
*** luisg has joined #openstack-infra | 09:22 | |
*** rpodolyaka has joined #openstack-infra | 09:22 | |
*** _cyril_ has joined #openstack-infra | 09:22 | |
*** mtreinish has joined #openstack-infra | 09:22 | |
*** yamahata__ has joined #openstack-infra | 09:22 | |
*** akscram has joined #openstack-infra | 09:22 | |
*** ilyashakhat has joined #openstack-infra | 09:22 | |
*** marun has joined #openstack-infra | 09:22 | |
*** zul has joined #openstack-infra | 09:22 | |
*** agordeev has joined #openstack-infra | 09:22 | |
*** avishayb has joined #openstack-infra | 09:22 | |
*** michchap has joined #openstack-infra | 09:22 | |
*** greghaynes has joined #openstack-infra | 09:22 | |
*** dkranz has joined #openstack-infra | 09:22 | |
*** mgagne has joined #openstack-infra | 09:22 | |
*** ianw has joined #openstack-infra | 09:22 | |
*** openstackgerrit has joined #openstack-infra | 09:22 | |
*** bnemec has joined #openstack-infra | 09:22 | |
*** bogdando has joined #openstack-infra | 09:22 | |
*** Hefeweizen has joined #openstack-infra | 09:22 | |
*** jerryz has joined #openstack-infra | 09:22 | |
*** fungi has joined #openstack-infra | 09:22 | |
*** shardy has joined #openstack-infra | 09:22 | |
*** Shrews has joined #openstack-infra | 09:22 | |
*** rockstar has joined #openstack-infra | 09:22 | |
*** jaypipes has joined #openstack-infra | 09:22 | |
*** portante has joined #openstack-infra | 09:22 | |
*** mordred has joined #openstack-infra | 09:22 | |
*** dansmith has joined #openstack-infra | 09:22 | |
*** dragondm has joined #openstack-infra | 09:22 | |
*** BobBall has joined #openstack-infra | 09:22 | |
*** jcooley has joined #openstack-infra | 09:22 | |
*** kiall has joined #openstack-infra | 09:22 | |
*** jeblair has joined #openstack-infra | 09:22 | |
*** samalba has joined #openstack-infra | 09:22 | |
*** koolhead17 has joined #openstack-infra | 09:22 | |
*** med_ has joined #openstack-infra | 09:22 | |
*** kpepple has joined #openstack-infra | 09:22 | |
*** ttx has joined #openstack-infra | 09:22 | |
*** stevebaker has joined #openstack-infra | 09:22 | |
*** adam_g has joined #openstack-infra | 09:22 | |
*** hub_cap has joined #openstack-infra | 09:22 | |
*** esmute has joined #openstack-infra | 09:22 | |
*** odyi has joined #openstack-infra | 09:22 | |
*** mika has joined #openstack-infra | 09:22 | |
*** jgriffith has joined #openstack-infra | 09:22 | |
*** EntropyWorks has joined #openstack-infra | 09:22 | |
*** SpamapS has joined #openstack-infra | 09:22 | |
*** pleia2 has joined #openstack-infra | 09:22 | |
*** basic` has joined #openstack-infra | 09:22 | |
*** thingee has joined #openstack-infra | 09:22 | |
*** hemanth_ has joined #openstack-infra | 09:22 | |
*** jpeeler has joined #openstack-infra | 09:22 | |
*** sdague has joined #openstack-infra | 09:22 | |
*** rainya has joined #openstack-infra | 09:22 | |
*** antonym has joined #openstack-infra | 09:22 | |
*** Mithrandir has joined #openstack-infra | 09:22 | |
*** vishy has joined #openstack-infra | 09:22 | |
*** zigo has joined #openstack-infra | 09:22 | |
*** Adri2000 has joined #openstack-infra | 09:22 | |
*** cyeoh has joined #openstack-infra | 09:22 | |
*** persia has joined #openstack-infra | 09:22 | |
*** morganfainberg has joined #openstack-infra | 09:22 | |
*** SlickNik has joined #openstack-infra | 09:22 | |
*** wayneeseguin has joined #openstack-infra | 09:22 | |
*** chmouel has joined #openstack-infra | 09:22 | |
*** metabro_ has joined #openstack-infra | 09:22 | |
*** rahmu has joined #openstack-infra | 09:22 | |
*** Daviey has joined #openstack-infra | 09:22 | |
*** spiffxp_ has joined #openstack-infra | 09:22 | |
*** mkerrin has joined #openstack-infra | 09:22 | |
*** yongli has joined #openstack-infra | 09:22 | |
*** dhellmann has joined #openstack-infra | 09:22 | |
*** torgomatic has joined #openstack-infra | 09:22 | |
*** danger_f` has joined #openstack-infra | 09:22 | |
*** Alex_Gaynor has joined #openstack-infra | 09:22 | |
*** annegentle_ has joined #openstack-infra | 09:22 | |
*** sirushti has joined #openstack-infra | 09:22 | |
*** davidlenwell has joined #openstack-infra | 09:22 | |
*** primeministerp has joined #openstack-infra | 09:22 | |
*** fbo_away has joined #openstack-infra | 09:22 | |
*** notmyname has joined #openstack-infra | 09:22 | |
*** devananda has joined #openstack-infra | 09:22 | |
*** gfa has joined #openstack-infra | 09:22 | |
*** jog0 has joined #openstack-infra | 09:22 | |
*** ewindisch has joined #openstack-infra | 09:22 | |
*** ogelbukh has joined #openstack-infra | 09:22 | |
*** adarazs has joined #openstack-infra | 09:22 | |
*** harlowja_away has joined #openstack-infra | 09:22 | |
*** enikanorov has joined #openstack-infra | 09:22 | |
*** lifeless has joined #openstack-infra | 09:22 | |
*** mkoderer_ has joined #openstack-infra | 09:22 | |
*** belliott has joined #openstack-infra | 09:22 | |
*** tsufiev_ has joined #openstack-infra | 09:22 | |
*** dstufft has joined #openstack-infra | 09:22 | |
*** kashyap has joined #openstack-infra | 09:22 | |
*** gaelL has joined #openstack-infra | 09:22 | |
*** tian has joined #openstack-infra | 09:22 | |
*** juice has joined #openstack-infra | 09:22 | |
*** guitarzan has joined #openstack-infra | 09:22 | |
*** GheRivero has joined #openstack-infra | 09:22 | |
*** iccha-dinner has joined #openstack-infra | 09:22 | |
*** StevenK has joined #openstack-infra | 09:22 | |
*** hughsaunders has joined #openstack-infra | 09:22 | |
*** maurosr has joined #openstack-infra | 09:22 | |
*** pabelanger has joined #openstack-infra | 09:22 | |
*** jd__ has joined #openstack-infra | 09:22 | |
*** saper has joined #openstack-infra | 09:22 | |
*** gema has joined #openstack-infra | 09:22 | |
*** ekarlso has joined #openstack-infra | 09:22 | |
*** ywu has joined #openstack-infra | 09:22 | |
*** Loquacities has joined #openstack-infra | 09:22 | |
*** Protux has joined #openstack-infra | 09:22 | |
*** olaph has joined #openstack-infra | 09:22 | |
*** Grishkin has joined #openstack-infra | 09:22 | |
*** pvo has joined #openstack-infra | 09:22 | |
*** Guest33069 has joined #openstack-infra | 09:22 | |
*** hdd has joined #openstack-infra | 09:22 | |
*** AaronGr_Zzz has joined #openstack-infra | 09:22 | |
*** mattymo has joined #openstack-infra | 09:22 | |
*** clarkb has joined #openstack-infra | 09:22 | |
*** zaro has joined #openstack-infra | 09:22 | |
*** sileht has joined #openstack-infra | 09:22 | |
*** mancdaz has joined #openstack-infra | 09:22 | |
*** milki has joined #openstack-infra | 09:22 | |
*** mindjiver has joined #openstack-infra | 09:22 | |
*** sbadia has joined #openstack-infra | 09:22 | |
*** jheiss has joined #openstack-infra | 09:22 | |
*** Ng has joined #openstack-infra | 09:22 | |
*** david-lyle has joined #openstack-infra | 09:22 | |
*** jlk has joined #openstack-infra | 09:22 | |
*** vladan has joined #openstack-infra | 09:22 | |
*** dachary has joined #openstack-infra | 09:22 | |
*** uvirtbot has joined #openstack-infra | 09:22 | |
*** EmilienM has joined #openstack-infra | 09:22 | |
*** russellb has joined #openstack-infra | 09:22 | |
*** jesusaurus has joined #openstack-infra | 09:22 | |
*** mikal has joined #openstack-infra | 09:22 | |
*** alaski has joined #openstack-infra | 09:22 | |
*** nikhil__ has joined #openstack-infra | 09:22 | |
*** anteaya has joined #openstack-infra | 09:22 | |
*** nijaba has joined #openstack-infra | 09:22 | |
*** westmau5 has joined #openstack-infra | 09:22 | |
*** jeremyb has joined #openstack-infra | 09:22 | |
*** ChanServ has joined #openstack-infra | 09:22 | |
*** hobana.freenode.net sets mode: +o ChanServ | 09:22 | |
*** crank has joined #openstack-infra | 09:22 | |
*** Ryan_Lane has quit IRC | 09:24 | |
*** crank has quit IRC | 09:27 | |
*** crank has joined #openstack-infra | 09:28 | |
*** jcooley_ has joined #openstack-infra | 09:30 | |
*** tsufiev_ has quit IRC | 09:31 | |
*** crank has quit IRC | 09:33 | |
*** crank has joined #openstack-infra | 09:34 | |
*** jcooley_ has quit IRC | 09:35 | |
*** fbo_away is now known as fbo | 09:38 | |
*** crank has quit IRC | 09:39 | |
*** ruhe has joined #openstack-infra | 09:46 | |
*** crank has joined #openstack-infra | 09:47 | |
*** crank has quit IRC | 09:52 | |
*** crank_ has joined #openstack-infra | 09:52 | |
*** fifieldt has joined #openstack-infra | 09:54 | |
*** crank_ has quit IRC | 09:57 | |
*** crank has joined #openstack-infra | 09:58 | |
*** fifieldt has quit IRC | 10:08 | |
*** jooools has joined #openstack-infra | 10:08 | |
*** crank has quit IRC | 10:12 | |
*** sarob has joined #openstack-infra | 10:12 | |
*** crank has joined #openstack-infra | 10:13 | |
*** rongze has quit IRC | 10:14 | |
*** dhellmann has quit IRC | 10:14 | |
*** mancdaz is now known as mancdaz_away | 10:15 | |
*** elasticio has joined #openstack-infra | 10:15 | |
*** dhellmann has joined #openstack-infra | 10:15 | |
*** sarob has quit IRC | 10:17 | |
*** crank has quit IRC | 10:17 | |
*** crank has joined #openstack-infra | 10:18 | |
*** jhesketh_ has joined #openstack-infra | 10:19 | |
*** Ryan_Lane has joined #openstack-infra | 10:19 | |
*** jhesketh_ has quit IRC | 10:20 | |
*** mancdaz_away is now known as mancdaz | 10:21 | |
*** ruhe has quit IRC | 10:23 | |
*** salv-orlando has quit IRC | 10:23 | |
*** jcooley_ has joined #openstack-infra | 10:23 | |
*** Ryan_Lane has quit IRC | 10:24 | |
*** jcooley_ has quit IRC | 10:28 | |
*** fifieldt has joined #openstack-infra | 10:30 | |
*** nati_ueno has quit IRC | 10:34 | |
*** yamahata has quit IRC | 10:42 | |
*** yamahata__ has quit IRC | 10:42 | |
*** yamahata__ has joined #openstack-infra | 10:42 | |
*** yamahata__ has quit IRC | 10:43 | |
*** yamahata__ has joined #openstack-infra | 10:43 | |
*** elasticio has quit IRC | 10:48 | |
*** crank has quit IRC | 10:49 | |
*** crank has joined #openstack-infra | 10:50 | |
*** yolanda has quit IRC | 10:51 | |
*** yolanda has joined #openstack-infra | 10:51 | |
*** fifieldt has quit IRC | 11:04 | |
*** ArxCruz has joined #openstack-infra | 11:04 | |
*** sarob has joined #openstack-infra | 11:12 | |
*** nati_ueno has joined #openstack-infra | 11:13 | |
*** SergeyLukjanov is now known as _SergeyLukjanov | 11:13 | |
*** _SergeyLukjanov has quit IRC | 11:14 | |
*** sarob has quit IRC | 11:17 | |
*** jcooley_ has joined #openstack-infra | 11:18 | |
*** crank has quit IRC | 11:18 | |
*** Ryan_Lane has joined #openstack-infra | 11:19 | |
*** crank has joined #openstack-infra | 11:19 | |
*** NikitaKonovalov has joined #openstack-infra | 11:20 | |
*** lcestari has joined #openstack-infra | 11:22 | |
*** jcooley_ has quit IRC | 11:23 | |
*** Ryan_Lane has quit IRC | 11:24 | |
*** rongze has joined #openstack-infra | 11:24 | |
*** mugsie_ has quit IRC | 11:24 | |
*** mugsie_ has joined #openstack-infra | 11:24 | |
*** mugsie_ is now known as mugsie | 11:24 | |
*** che-arne has joined #openstack-infra | 11:25 | |
*** elasticio has joined #openstack-infra | 11:35 | |
*** guohliu has quit IRC | 11:36 | |
*** chenglin has joined #openstack-infra | 11:44 | |
*** marun has quit IRC | 11:48 | |
*** praneshp has quit IRC | 11:51 | |
*** chenglin has quit IRC | 11:52 | |
*** chenglin has joined #openstack-infra | 11:52 | |
*** chenglin has quit IRC | 11:53 | |
*** nosnos has quit IRC | 11:53 | |
*** plomakin has joined #openstack-infra | 11:53 | |
*** chenglin has joined #openstack-infra | 11:54 | |
*** chenglin has quit IRC | 11:54 | |
*** chenglin has joined #openstack-infra | 11:54 | |
*** chenglin has quit IRC | 11:54 | |
*** vogxn has joined #openstack-infra | 11:55 | |
*** lcheng has joined #openstack-infra | 11:55 | |
*** elasticio has quit IRC | 11:55 | |
*** crank has quit IRC | 12:00 | |
*** talluri has quit IRC | 12:01 | |
*** afazekas_ has joined #openstack-infra | 12:01 | |
*** crank has joined #openstack-infra | 12:01 | |
*** talluri has joined #openstack-infra | 12:01 | |
*** dolphm has joined #openstack-infra | 12:02 | |
*** CaptTofu has quit IRC | 12:04 | |
*** CaptTofu has joined #openstack-infra | 12:04 | |
BobBall | sdague: Regarding nose rename of create_test_server - I was requested to restore the change by matthew treinish due to people trying to increase the quotas to fix a different test. | 12:05 |
*** talluri has quit IRC | 12:06 | |
BobBall | sdague: I'm more than happy for the change to be abandoned (as it had been before) - but do you have an issue with just renaming the method? or is the issue the reason that I want to rename it? | 12:06 |
*** markmc has joined #openstack-infra | 12:08 | |
*** zhiyan has quit IRC | 12:08 | |
*** jcooley_ has joined #openstack-infra | 12:11 | |
*** rfolco has joined #openstack-infra | 12:11 | |
*** sarob has joined #openstack-infra | 12:12 | |
*** zhiyan has joined #openstack-infra | 12:13 | |
*** miqui has quit IRC | 12:13 | |
*** yaguang has quit IRC | 12:16 | |
*** salv-orlando has joined #openstack-infra | 12:16 | |
*** jcooley_ has quit IRC | 12:17 | |
*** Ryan_Lane has joined #openstack-infra | 12:19 | |
*** tma996 has joined #openstack-infra | 12:19 | |
*** krtaylor has joined #openstack-infra | 12:21 | |
*** avishayb has quit IRC | 12:23 | |
*** Ryan_Lane has quit IRC | 12:24 | |
*** smarcet has joined #openstack-infra | 12:29 | |
*** avishayb has joined #openstack-infra | 12:31 | |
*** crank has quit IRC | 12:34 | |
*** Abhishek has joined #openstack-infra | 12:34 | |
*** crank has joined #openstack-infra | 12:34 | |
*** sandywalsh_ has quit IRC | 12:36 | |
sdague | so we keep putting bandaids around nose | 12:36 |
sdague | and we need to just fix whatever is keeping people from using testr | 12:36 |
sdague | and explode if they try to run under nose | 12:36 |
BobBall | I understand that - but last time I talked with _someone_ (can't remember who) the suggestion was that we can't deprecate nose for a while? | 12:37 |
BobBall | I figured that this change - even though it's only a bandaid - was a good thing to have to save the current frustrations | 12:38 |
*** vogxn has quit IRC | 12:39 | |
*** dolphm has quit IRC | 12:39 | |
*** dolphm has joined #openstack-infra | 12:41 | |
*** thomasem has joined #openstack-infra | 12:43 | |
*** zhiyan has left #openstack-infra | 12:43 | |
*** sarob has quit IRC | 12:45 | |
*** crank has quit IRC | 12:48 | |
sdague | so the problem is it's not trivial, but lifeless had a path to get us there | 12:49 |
*** sandywalsh_ has joined #openstack-infra | 12:49 | |
*** crank has joined #openstack-infra | 12:49 | |
*** yolanda has quit IRC | 12:50 | |
sdague | so I'm just putting my foot down and someone should work on it, no new nose fixes. | 12:50 |
*** yolanda has joined #openstack-infra | 12:50 | |
sdague | because nose working in the tree actually means we get tests in that aren't parallel safe | 12:50 |
sdague | and I've had to revert a couple of those | 12:50 |
BobBall | OK - perhaps an alternative change would be to require the "--use-deprecated-nose" flag or abort ;) | 12:51 |
BobBall | I'll abandon my change | 12:52 |
BobBall | I'm just not sure that most of the active people will care much because "everyone uses tox" | 12:52 |
*** dprince has joined #openstack-infra | 12:59 | |
sdague | man grizzly changes are just full of fails | 13:00 |
freyes | hello everybody | 13:06 |
*** jcooley_ has joined #openstack-infra | 13:07 | |
freyes | I abandoned this patch https://review.openstack.org/#/c/61547/ in favor of this one https://review.openstack.org/#/c/60066/, but we haven't had any news since a couple of weeks, what's the recommended time to wait for updates? | 13:10 |
sdague | freyes: yuo had -1 feedback from Kai on Dec 5 which was not addressed | 13:12 |
*** xchu has joined #openstack-infra | 13:12 | |
*** sarob has joined #openstack-infra | 13:12 | |
sdague | basically, if a patch has -1 on it, it's going to be largely ignored by other folks, as everyone has a lot on their review queue. So address those comments and upload a new change | 13:13 |
*** jcooley_ has quit IRC | 13:13 | |
*** dkranz has quit IRC | 13:16 | |
*** sarob has quit IRC | 13:16 | |
freyes | sdague, I abandoned my patch, because mathieu ask me to do it, 60066 arrived before and we didn't want to duplicate efforts | 13:18 |
*** psedlak_clone is now known as psedlak | 13:18 | |
*** rongze has quit IRC | 13:19 | |
*** Ryan_Lane has joined #openstack-infra | 13:19 | |
*** Protux has quit IRC | 13:19 | |
*** eharney has joined #openstack-infra | 13:19 | |
*** rongze has joined #openstack-infra | 13:21 | |
*** freyes has quit IRC | 13:23 | |
*** freyes has joined #openstack-infra | 13:23 | |
*** mriedem has joined #openstack-infra | 13:23 | |
*** Ryan_Lane has quit IRC | 13:24 | |
*** esker has quit IRC | 13:26 | |
*** esker has joined #openstack-infra | 13:27 | |
*** herndon has joined #openstack-infra | 13:28 | |
*** dolphm has quit IRC | 13:28 | |
*** dolphm has joined #openstack-infra | 13:30 | |
*** esker has quit IRC | 13:31 | |
*** spredzy_ is now known as spredzy | 13:42 | |
*** beagles is now known as beagles_brb | 13:48 | |
*** NikitaKonovalov has quit IRC | 13:50 | |
*** NikitaKonovalov has joined #openstack-infra | 13:50 | |
*** SergeyLukjanov has joined #openstack-infra | 13:51 | |
*** jpich has quit IRC | 13:52 | |
*** jpich has joined #openstack-infra | 13:53 | |
*** che-arne has quit IRC | 13:53 | |
sdague | freyes: so I would try to ping stewart, and if nothing else you can take over that patch (git review -d ) and make the fixes needed | 13:53 |
*** che-arne has joined #openstack-infra | 13:53 | |
freyes | sdague, thanks :) | 13:53 |
*** amotoki has joined #openstack-infra | 13:55 | |
*** mattymo has quit IRC | 13:55 | |
*** mattymo has joined #openstack-infra | 13:56 | |
*** gothicmindfood has joined #openstack-infra | 13:58 | |
*** julim has joined #openstack-infra | 13:58 | |
*** jcooley_ has joined #openstack-infra | 14:00 | |
*** gothicmindfood has quit IRC | 14:04 | |
*** jcooley_ has quit IRC | 14:06 | |
*** yaguang has joined #openstack-infra | 14:07 | |
*** zul has quit IRC | 14:09 | |
*** weshay has joined #openstack-infra | 14:09 | |
*** dkranz has joined #openstack-infra | 14:09 | |
*** CaptTofu has quit IRC | 14:10 | |
*** CaptTofu has joined #openstack-infra | 14:10 | |
*** nati_ueno has quit IRC | 14:10 | |
*** CaptTofu has quit IRC | 14:11 | |
*** zul has joined #openstack-infra | 14:11 | |
*** sarob has joined #openstack-infra | 14:12 | |
*** plomakin has quit IRC | 14:12 | |
openstackgerrit | Salvatore Orlando proposed a change to openstack-infra/elastic-recheck: Add query for bug 1254890 https://review.openstack.org/62891 | 14:13 |
uvirtbot | Launchpad bug 1254890 in tempest ""Timed out waiting for thing" causes tempest-dsvm-neutron-* failures" [Low,In progress] https://launchpad.net/bugs/1254890 | 14:13 |
*** herndon has quit IRC | 14:15 | |
*** mfer has joined #openstack-infra | 14:15 | |
*** dstanek has joined #openstack-infra | 14:17 | |
*** sarob has quit IRC | 14:17 | |
openstackgerrit | A change was merged to openstack-infra/elastic-recheck: Add query for bug 1254890 https://review.openstack.org/62891 | 14:23 |
uvirtbot | Launchpad bug 1254890 in tempest ""Timed out waiting for thing" causes tempest-dsvm-neutron-* failures" [Low,In progress] https://launchpad.net/bugs/1254890 | 14:23 |
*** oubiwann has joined #openstack-infra | 14:25 | |
openstackgerrit | A change was merged to openstack-infra/elastic-recheck: Machine remins in HARD_REBOOT status https://review.openstack.org/61815 | 14:25 |
*** dizquierdo has joined #openstack-infra | 14:28 | |
*** yassine has quit IRC | 14:33 | |
*** yassine has joined #openstack-infra | 14:33 | |
*** dkliban has joined #openstack-infra | 14:39 | |
flaper87 | guys, any chance I can get your help on figuring out what's happening here? https://review.openstack.org/#/c/32544/ | 14:40 |
flaper87 | For some reason, that test fails in the gate but neither me nor other guys I've asked can replicate it | 14:41 |
*** dims_ has quit IRC | 14:44 | |
fungi | flaper87: unit.cache.test_cache.TestCacheModule.test_get_cache_options fails on assertEqual(driver._options, {'default_ttl': '4'}) with MismatchError: {} != {'default_ttl': '4'} | 14:47 |
fungi | flaper87: you're unable to reproduce that under python 2.7 on ubuntu 12.04? | 14:47 |
fungi | i can try and see what i get | 14:48 |
*** dims_ has joined #openstack-infra | 14:48 | |
flaper87 | I tested it on f19 and Alejandro tested it on Ubuntu (IIRC) | 14:49 |
*** jlibosva has joined #openstack-infra | 14:49 | |
*** SushilKM has quit IRC | 14:49 | |
*** johnthetubaguy has joined #openstack-infra | 14:49 | |
flaper87 | fungi: if you can give it a try, I'd really appreciate it | 14:49 |
anteaya | jlibosva: here you are | 14:51 |
anteaya | fungi: when you have a minute, jlibosva and I were wondering if you can ssh into a vm running his patch and do some exploration | 14:52 |
anteaya | it is the neutron grenade patch and he is unable to replicate failures locally | 14:52 |
fungi | anteaya: perhaps. what is it a patch of/for? | 14:52 |
fungi | you anticipate my questions before i can ask them ;) | 14:52 |
anteaya | jlibosva: can we have the patch url please? | 14:53 |
anteaya | fungi: I try | 14:53 |
* jlibosva searching | 14:53 | |
jlibosva | https://review.openstack.org/#/c/58695/ | 14:53 |
fungi | flaper87: i've got the offending testset running in a vm here... i'll let you know what i get | 14:53 |
fungi | jlibosva: okay, so it looks like grenade is setting up an initial stable/havana devstack with neutron in preparation for your change, but devstack exercises cannot complete | 14:57 |
fungi | is it possible devstack with neutron is broken in havana presently? | 14:57 |
jlibosva | idk, are there still running any tests with havana? | 14:58 |
fungi | sdague: ^ do you know about any possible havana devstack+neutron issues at the moment? | 14:58 |
*** yolanda has quit IRC | 15:00 | |
fungi | specifically, devstack exercises failing on [ERROR] /opt/stack/old/devstack/lib/neutron:824 [Fail] Couldn't ping server | 15:00 |
*** yolanda has joined #openstack-infra | 15:01 | |
fungi | jlibosva: anteaya: i'm leaning toward it being bug 1261182 | 15:01 |
uvirtbot | Launchpad bug 1261182 in neutron "[ERROR] /opt/stack/old/devstack/lib/neutron:824 [Fail] Couldn't ping server" [Undecided,New] https://launchpad.net/bugs/1261182 | 15:01 |
*** dcramer_ has joined #openstack-infra | 15:02 | |
jlibosva | damn, why am I spending so much time debugging when it's already reported? :D | 15:02 |
*** esker has joined #openstack-infra | 15:02 | |
jlibosva | ah, it's not debugged yet, so maybe it was worth it :) | 15:02 |
anteaya | fungi: you seem to have found the issue | 15:03 |
anteaya | jlibosva: you have exercises working locally, correct? | 15:03 |
jlibosva | fungi: I tried that locally on my environment and I'm suspecting that ovs is not used correctly, I'd like to see whether it behaves the same on gate job | 15:03 |
jlibosva | anteaya: no, my floating_ip exercise fails too | 15:03 |
*** beagles_brb is now known as beagles | 15:04 | |
*** flaper87 is now known as flaper87|afk | 15:04 | |
anteaya | jlibosva: ah | 15:04 |
* fungi looks at the floating_ip results | 15:04 | |
fungi | yeah, same behavior though | 15:04 |
jlibosva | anteaya: but I'm not sure I understand that correctly. I see that interfaces in qdhcp and qrouter uses tag 1 , but instance uses tag 2 | 15:05 |
jlibosva | that could explain why dhcp packets cannot reach dhcp | 15:05 |
fungi | jlibosva: keep in mind that step #1 in grenade is to set up the previous stable release in devstack and make sure it's operable (so your patch isn't even applied at the point where that's failing) | 15:05 |
*** NikitaKonovalov has quit IRC | 15:06 | |
fungi | that's a safety valve to keep us from testing an upgrade from a broken stable state | 15:06 |
jlibosva | fungi: I'm aware of that :) But I was asked to have exercises running before upgrade | 15:06 |
*** SergeyLukjanov has quit IRC | 15:06 | |
fungi | what i'm saying is, i don't see any way this bug is related to your patch. if devstack exercises at the start of grenade for a change to master are failing, that indicates a need to fix something in the stable/havana branch | 15:08 |
*** miqui has joined #openstack-infra | 15:08 | |
*** rongze has quit IRC | 15:10 | |
*** miqui has quit IRC | 15:10 | |
*** flaper87|afk is now known as flaper87 | 15:11 | |
*** miqui has joined #openstack-infra | 15:12 | |
*** sarob has joined #openstack-infra | 15:12 | |
*** rcleere has joined #openstack-infra | 15:14 | |
fungi | i'd suggest rechecking against 1261182 and see if grenade will get any farther on the next run so we can see if your patch is doing what you intend | 15:15 |
fungi | but as far as broken devstack exercises, those are broken in stable/havana on that test run | 15:15 |
*** ruhe has joined #openstack-infra | 15:16 | |
fungi | (the *current* state of the stable/havana branch tip, unrelated to your change to master) | 15:16 |
*** sarob has quit IRC | 15:16 | |
*** pblaho has quit IRC | 15:17 | |
anteaya | fungi: thanks for the help | 15:18 |
fungi | flaper87: on a debian/testing vm i had sitting around, your change fails under a python 2.7 tox environment for me at unit.reports.test_guru_meditation_report.TestGuruMeditationReport.test_bas | 15:18 |
fungi | ic_report | 15:18 |
anteaya | I'll follow up with -qa and see if we can keep jlibosva moving forward | 15:18 |
fungi | flaper87: i'll build an ubuntu vm and see what i get there | 15:18 |
*** Ryan_Lane has joined #openstack-infra | 15:19 | |
flaper87 | fungi: mmh, weird. Pls, let me know! THanks for the support! | 15:20 |
*** talluri has joined #openstack-infra | 15:24 | |
*** wenlock has joined #openstack-infra | 15:24 | |
*** rnirmal has joined #openstack-infra | 15:24 | |
*** herndon has joined #openstack-infra | 15:26 | |
*** ruhe has quit IRC | 15:30 | |
*** wenlock has quit IRC | 15:32 | |
*** sdake_ has joined #openstack-infra | 15:33 | |
*** talluri_ has joined #openstack-infra | 15:34 | |
*** mrodden has joined #openstack-infra | 15:34 | |
*** talluri has quit IRC | 15:35 | |
*** xchu has quit IRC | 15:36 | |
*** UtahDave has joined #openstack-infra | 15:36 | |
*** SergeyLukjanov has joined #openstack-infra | 15:37 | |
*** CaptTofu has joined #openstack-infra | 15:37 | |
*** Ryan_Lane has quit IRC | 15:38 | |
*** davidhadas has quit IRC | 15:39 | |
*** senk has joined #openstack-infra | 15:40 | |
*** vkozhukalov has quit IRC | 15:40 | |
*** rongze has joined #openstack-infra | 15:40 | |
*** agordeev has quit IRC | 15:44 | |
*** senk has quit IRC | 15:45 | |
*** jcooley_ has joined #openstack-infra | 15:48 | |
*** dkehn has quit IRC | 15:48 | |
*** david-lyle_ has joined #openstack-infra | 15:49 | |
*** dkehn has joined #openstack-infra | 15:50 | |
*** senk has joined #openstack-infra | 15:52 | |
*** david-lyle_ has quit IRC | 15:53 | |
fungi | flaper87: i'm able to perfectly reproduce the error in your unit test log | 15:53 |
*** david-lyle_ has joined #openstack-infra | 15:53 | |
flaper87 | fungi: mmh | 15:53 |
flaper87 | fungi: ubuntu? What python version? | 15:53 |
fungi | flaper87: i built an ubuntu 12.04 vm in rackspace following the instructions at https://git.openstack.org/cgit/openstack-infra/devstack-gate/tree/README.rst#n100 and then installed the needed libmysqlclient-dev package | 15:54 |
*** jcooley_ has quit IRC | 15:54 | |
*** johnthetubaguy1 has joined #openstack-infra | 15:55 | |
fungi | flaper87: then checked out oslo-incubator master branch from git, cherry-picked your change onto it, and ran tox -epy27 | 15:55 |
flaper87 | fungi: is there a chance I can access that box you created? | 15:55 |
flaper87 | It'd make it easier to figure out what's happening and why | 15:55 |
flaper87 | I can't replicated it in my f19 box | 15:55 |
flaper87 | :/ | 15:55 |
*** johnthetubaguy has quit IRC | 15:55 | |
*** rongze has quit IRC | 15:55 | |
fungi | flaper87: sure, is your ssh public key on your lp account profile? | 15:55 |
flaper87 | fungi: yes sir | 15:56 |
fungi | flaper87: you should be able to ssh into jenkins@162.209.101.7 or jenkins@2001:4802:7800:1:3bc3:d7f6:ff20:2f46 with that key | 15:57 |
fungi | the oslo-incubator checkout i was testing with is in the jenkins homedir | 15:57 |
flaper87 | fungi: I'm in, thanks a lot! | 15:57 |
fungi | let me know when you're done so i can discard that vm | 15:57 |
flaper87 | fungi: I'll let you know when I'm done with it | 15:57 |
flaper87 | hehe | 15:57 |
fungi | perfect | 15:58 |
*** mrodden has quit IRC | 16:01 | |
russellb | were is the code for status.openstack.org/zuul/ ? | 16:03 |
russellb | where | 16:03 |
russellb | i'm interested in trying to add a compact display option ... | 16:03 |
openstackgerrit | Jakub Libosvar proposed a change to openstack-infra/config: Adding non-voting neutron exercises gate job https://review.openstack.org/62924 | 16:04 |
fungi | russellb: i would love that--i've wanted one as well and just haven't had the time (something to collapse/expand one or all changes to/from just the heading bars?) http://git.openstack.org/cgit/openstack-infra/config/tree/modules/openstack_project/files/zuul | 16:05 |
fungi | the status.* files are the ones you want, i think | 16:05 |
sdague | yeh status.html is the main page | 16:05 |
russellb | fungi: yeah that's what i was thinking | 16:05 |
sdague | all the real work is in https://github.com/openstack-infra/config/blob/master/modules/openstack_project/files/zuul/status.js | 16:06 |
russellb | i have 0 web skill ... sounds like a good way to learn something, heh | 16:06 |
russellb | thanks guys | 16:06 |
fungi | thank YOU! | 16:06 |
russellb | thank me if i do it | 16:06 |
fungi | heh | 16:06 |
russellb | but i plan to try | 16:06 |
sdague | russellb: basically I think yuo want to write an alternate format_change | 16:07 |
*** ryanpetrello has joined #openstack-infra | 16:07 | |
sdague | and do a conditional around that | 16:07 |
*** Abhishek has quit IRC | 16:07 | |
sdague | if you can slice it at that level, it won't be too bad, if yuo need to go above that level, things will get more complicated | 16:07 |
*** jlibosva has quit IRC | 16:08 | |
UtahDave | fungi: Hey, I've got an example of using the Salt reactor at https://etherpad.openstack.org/p/Salt-Event-System | 16:09 |
fungi | UtahDave: great! taking a look now | 16:10 |
UtahDave | Let me know when you have a minute to go over it | 16:10 |
jeblair | russellb: oh hey | 16:10 |
russellb | ohai | 16:10 |
jeblair | russellb: i actually designed the current layout to work with the change box collapsed | 16:10 |
jeblair | russellb: i just didn't get around to that... | 16:10 |
russellb | nice | 16:10 |
*** blamar has quit IRC | 16:10 | |
*** SushilKM__ has joined #openstack-infra | 16:10 | |
russellb | yeah, i just want a way to have it collapsed by default, and have a little thingy i can click to expand for changes i care about | 16:10 |
jeblair | russellb: so if you make it so that the part below the box header isn't drawn | 16:10 |
russellb | or something like that | 16:10 |
jeblair | russellb: it should still work -- the subway map and everything | 16:10 |
russellb | woot | 16:11 |
jeblair | russellb: sounds perfect! | 16:11 |
fungi | UtahDave: i'll spend a few minutes digesting it and then ask the myriad of questions i'm still likely to harbor | 16:11 |
russellb | think it could be collapsed by default? or would people be annoyed | 16:11 |
UtahDave | Sounds good. I'm going to add some notes as well. | 16:11 |
russellb | so it doesn't have to be a toggle somewhere | 16:11 |
russellb | meh, i'll figure something out | 16:11 |
*** sarob has joined #openstack-infra | 16:12 | |
jeblair | russellb: i think ability to toggle individual and toggle all would be nice, and collapsed by default is good | 16:12 |
russellb | ack | 16:12 |
jeblair | russellb: (i think infra/qa people are likely the folks who want to see whole system status, so we should support that but it doesn't need to be default) | 16:13 |
hashar | jeblair: hey, I am looking at the Zuul templates changes you made recently :) | 16:13 |
jeblair | hashar: hey! | 16:13 |
*** yamahata has joined #openstack-infra | 16:13 | |
*** dcramer_ has quit IRC | 16:13 | |
fungi | jeblair: russellb: the other usability thing i've wanted is a highlighter similar to the project filter but extended to also match changes/gitshas and make matching items visually distinct (rather than hiding the others) so you can find/follow them when it's busy | 16:15 |
jeblair | fungi: yeah, that'd be nice | 16:15 |
*** jcooley_ has joined #openstack-infra | 16:15 | |
fungi | maybe bolding the heading text or changing the fill tint | 16:15 |
*** tma996 has quit IRC | 16:16 | |
russellb | yet another thing i'd like is a 2nd line in the header bar with the change summary line :) | 16:17 |
russellb | as that's how i generally remember patches instead of review numbers | 16:17 |
*** sarob has quit IRC | 16:17 | |
russellb | oh the possibilities! | 16:17 |
jeblair | russellb: how human of you. upgrade your cyborg implant! | 16:18 |
fungi | inferior thinkmeat | 16:18 |
*** ruhe has joined #openstack-infra | 16:19 | |
*** Abhishek has joined #openstack-infra | 16:20 | |
jeblair | they're made out of meat. | 16:21 |
fungi | frightening | 16:21 |
*** thomasbiege1 has joined #openstack-infra | 16:23 | |
*** markmcclain has quit IRC | 16:25 | |
*** ruhe has quit IRC | 16:27 | |
*** thomasbiege has quit IRC | 16:27 | |
*** dcramer_ has joined #openstack-infra | 16:27 | |
*** SushilKM__ has quit IRC | 16:27 | |
*** yassine has quit IRC | 16:28 | |
*** yassine has joined #openstack-infra | 16:29 | |
hashar | jeblair: left you a long meaningless comment on the Zuul template change https://review.openstack.org/#/c/62498/ | 16:30 |
hashar | getting fresh air be back in a few :-] | 16:31 |
jeblair | hashar: cool, thanks. i didn't think of that situation. | 16:31 |
ArxCruz | jeblair: fungi do you know why I'm getting this error? ERROR zuul.Gearman: Job <gear.Job 0x7fec98209810 handle: None name: build:gate-heat-python26 unique: 4525449a714049d0b2c1054397621852> is not registered with Gearman | 16:32 |
jeblair | ArxCruz: are you using jenkins with the gearman plugin? | 16:33 |
ArxCruz | jeblair: yes | 16:33 |
*** yaguang has quit IRC | 16:33 | |
jeblair | ArxCruz: that means it hasn't registered that job (or 'project' in jenkins-speak); either because the job doesn't exist in jenkins or some other reason (possibly a bug) | 16:34 |
sdague | jeblair: one other thing that occurred to me this morning, in dealing with a gate reset. We are currently putting grizzly changes into the same gate as master changes, however there is actually no link between those (no overlapping grenade job). How hard would it be to have zuul understand that? | 16:34 |
jeblair | ArxCruz: do any jobs work or is it just that one? | 16:34 |
ArxCruz | right now the gate-heat-python27 start works | 16:34 |
jeblair | ArxCruz: oh, another possibility is that there is no jenkins node currently capable of running it -- if that's the case, gearman-plugin won't register it | 16:34 |
*** mrodden has joined #openstack-infra | 16:34 | |
fungi | sdague: we're putting grizzly changes in the same gate as havana changes and havana changes in the same gate as master changes | 16:34 |
ArxCruz | jeblair: oh, you mean, the number of executors is full ? | 16:35 |
fungi | sdague: also, grizzly branches of server projects interact with master branches of client libraries | 16:35 |
*** SushilKM__ has joined #openstack-infra | 16:35 | |
sdague | fungi: if any of those are in the gate | 16:35 |
*** dkranz has quit IRC | 16:35 | |
jeblair | ArxCruz: no, i mean that there are no executors; for instance, we run 2.6 jobs on centos nodes, so there have to be centos nodes available for the job to be registered initially | 16:35 |
fungi | sdague: true | 16:35 |
sdague | the instance I had this morning was a bunch of tempest master changes, and a heat grizzly one was in the queue | 16:36 |
sdague | which blew up on a race | 16:36 |
ArxCruz | jeblair: fair enough, I have only one node added now :) | 16:36 |
sdague | I didn't think I had havana tempest in there.... but maybe I did | 16:36 |
openstackgerrit | Anita Kuno proposed a change to openstack-infra/config: Turn on devstack exercises for neutron (non-voting) https://review.openstack.org/62930 | 16:36 |
jeblair | ArxCruz: one node should be enough for it to register; you might try taking the node offline and putting it back online again. sometimes we see gearman-plugin not register. :( | 16:36 |
*** pcrews has joined #openstack-infra | 16:37 | |
jeblair | ArxCruz: you can telnet to gearman on port 4730 and type 'status' to see the list of registered jobs | 16:37 |
fungi | sdague: i think shared queue definition would have to become dynamic if it were to depend on which changes were in the queue at some point in time, and then how do you merge two separate queues when a change which needs to be in both of them enters the gate | 16:37 |
sdague | I guess if we just had master tempest and grizzly heat, would they form 2 queues? | 16:37 |
hashar | jeblair: merging jobs trees is a bit tricky but I am sure python has a solution build-in :-] | 16:37 |
jeblair | hashar: i'm still thinking about it, but i'm not sure that one way or the other is correct... | 16:37 |
sdague | fungi: right, so in that case it would have to wait to enter until the queues ahead resolved | 16:38 |
sdague | yeh, it gets complicated | 16:38 |
sdague | ok, just thinking aloud | 16:38 |
*** dstanek has quit IRC | 16:38 | |
*** yassine has quit IRC | 16:38 | |
fungi | sdague: i think it's probably solvable, but at the cost of a lot of complexity (bringing risk of new hard-to-diagnose bugs) and time probably better spent elsewhere | 16:39 |
sdague | yep, definitely | 16:39 |
sdague | thanks for explaining the complexity of it | 16:39 |
sdague | hadn't thought it all through | 16:39 |
ArxCruz | jeblair: cool, thanks | 16:39 |
*** yassine has joined #openstack-infra | 16:39 | |
jeblair | hashar: i'm thinking maybe the default should be what's in there now (don't collapse), and maybe add an option to the template "collapse-parents: True" to toggle the behavior you described | 16:40 |
jeblair | hashar: and i'm thinking maybe not writing that right now, but documenting the current behavior and leaving a TODO note. :) | 16:40 |
hashar | jeblair: lazy guy :-D :-D | 16:40 |
*** sdake has quit IRC | 16:41 | |
hashar | jeblair: that would work. The code is fine anyway, I did test it out with my layout and it works. Not sure what happens when a job is referenced twice though, I guess it would be triggered two times. | 16:41 |
*** krotscheck_nexus has joined #openstack-infra | 16:41 | |
*** sdake has joined #openstack-infra | 16:41 | |
hashar | jeblair: I might give a shot at merging the job trees | 16:41 |
jeblair | hashar: yeah, i recently made it so that you could list a job twice and it would run twice (we wanted that as a tool to deal with nondeterministic bugs), which is why i think both behaviors are desirable | 16:42 |
hashar | jeblair: nasty but valid use case :-] | 16:43 |
hashar | so yeah merging should be done if and only if it got requested | 16:43 |
jeblair | hashar: that's what i'm thinking; not merging seems to match the current behavior so should probably be the default | 16:43 |
*** romcheg1 has joined #openstack-infra | 16:44 | |
jeblair | hashar: cool, i'll go make doc/TODO updates | 16:44 |
romcheg1 | Hi folks | 16:44 |
*** romcheg1 is now known as romcheg | 16:44 | |
hashar | jeblair: will not be connected tonight, so will only review that tomorrow. But if doc/TODO is around I guess it is good for me | 16:45 |
hashar | jeblair: the following change (provide 'name' to templates) is fine to me though haven't tested it | 16:45 |
hashar | I +1ed it already | 16:45 |
*** krotscheck_nexus has quit IRC | 16:45 | |
*** krotscheck_nexus has joined #openstack-infra | 16:45 | |
romcheg | As I see some Neutron jobs define DEVSTACK_GATE_SMOKE_SERIAL=1 flag because there was a bug that failed to run Neutron tests in parallel. | 16:46 |
jeblair | hashar: cool, i think it would only be a problem if you used 'name' as a substitution variable (and used it to mean something other than what zuul is going to set it to) | 16:46 |
romcheg | I found that the bug is in Fix released state | 16:46 |
romcheg | Can I add my new neutron job without setting that flag? | 16:47 |
jeblair | romcheg: nice. i'm not fully familiar with this; you might want to check with sdague to see if he knows of any other reasons it should still be set... | 16:48 |
romcheg | jeblair: thanks. | 16:48 |
romcheg | sdague: Are you around? | 16:48 |
jeblair | romcheg: also, you can propose an experimental change to devstack-gate that just ignores that flag; that will run all of the jobs and you can see how well it works | 16:49 |
*** senk has quit IRC | 16:49 | |
jeblair | romcheg: that's a quick way to check what will happen without having to define a new job | 16:49 |
romcheg | jeblair: That's what I did. Just want to get more details | 16:49 |
*** krotscheck_nexus has quit IRC | 16:50 | |
*** dkranz has joined #openstack-infra | 16:50 | |
hashar | jeblair: I use "prefix" for the Zuul job-template :-] | 16:53 |
*** yassine has quit IRC | 16:53 | |
*** avishayb has quit IRC | 16:55 | |
*** rongze has joined #openstack-infra | 16:55 | |
*** smarcet has quit IRC | 16:57 | |
openstackgerrit | Roman Prykhodchenko proposed a change to openstack-infra/config: Adds devstack-gate tests for Ironic https://review.openstack.org/53917 | 16:57 |
openstackgerrit | Roman Prykhodchenko proposed a change to openstack-infra/config: Add devstack-gate jobs for Ironic https://review.openstack.org/53917 | 17:00 |
*** jgriffith has quit IRC | 17:02 | |
*** dstanek has joined #openstack-infra | 17:04 | |
*** krotscheck has joined #openstack-infra | 17:04 | |
*** davidhadas has joined #openstack-infra | 17:06 | |
*** jcoufal has quit IRC | 17:06 | |
*** sarob has joined #openstack-infra | 17:07 | |
*** rwsu has joined #openstack-infra | 17:08 | |
*** mancdaz is now known as mancdaz_away | 17:08 | |
*** dstanek has quit IRC | 17:09 | |
fungi | UtahDave: i've added some comments to the end of that etherpad i'd like to get your feedback on when you have time | 17:12 |
UtahDave | fungi: sure! | 17:13 |
*** agordeev has joined #openstack-infra | 17:17 | |
*** blamar has joined #openstack-infra | 17:18 | |
openstackgerrit | Anita Kuno proposed a change to openstack-infra/config: Turn on devstack exercises for neutron (non-voting) https://review.openstack.org/62930 | 17:19 |
*** SushilKM__ has quit IRC | 17:20 | |
*** SushilKM has joined #openstack-infra | 17:20 | |
devananda | clarkb: ping | 17:22 |
*** fbo is now known as fbo_away | 17:27 | |
clarkb | devananda: pong | 17:30 |
*** dkranz has quit IRC | 17:30 | |
zaro | good morning | 17:31 |
*** senk1 has joined #openstack-infra | 17:31 | |
devananda | clarkb: following up on a chat romcheg and I were just having -- the plan for ironic/tempest stuff. I thought there was concensus, but now i'm not sure what order we should be adding the tests where... | 17:32 |
*** ruhe has joined #openstack-infra | 17:33 | |
fungi | zaro: since we said we'd approve 61542 if we didn't get mordred's feedback by last night, i went ahead and did | 17:33 |
clarkb | devananda: I think you start with devstack-gate job where ironic gets installed and configured. This requires the creation of "baremetal" VMs and the enrollment of them into ironic | 17:33 |
clarkb | devananda: then you can have tempest tests do whatever with those resources | 17:33 |
*** vkozhukalov has joined #openstack-infra | 17:34 | |
fungi | zaro: i want to try manually rolling back review-dev to our 2.4.4 fork briefly just to see if it works (and to test a link regex change on it real quick), if that's cool with you | 17:35 |
openstackgerrit | A change was merged to openstack-infra/config: give zaro access to review-dev server https://review.openstack.org/61542 | 17:35 |
romcheg | clarkb: I think the concern it about the pipeline | 17:35 |
zaro | fungi: no problem, take your time. | 17:35 |
clarkb | romcheg: devananda: what is the concern? | 17:35 |
devananda | clarkb: two different types of testing -- i'm referring to API testing by tempest, romcheg's patches that have been up for a few weeks | 17:36 |
clarkb | devananda: I don't think they are any different they both require ironic to be running right? | 17:36 |
devananda | clarkb: not the "devstack-gate spins up a cloud using ironic" tests | 17:36 |
romcheg | In this patch I add these baremetal jobs to the experimental branch https://review.openstack.org/#/c/53917/ | 17:36 |
openstackgerrit | James E. Blair proposed a change to openstack-infra/zuul: Allow layered templates https://review.openstack.org/62498 | 17:36 |
devananda | clarkb: they are very different :-/ | 17:36 |
clarkb | devananda: but not from an infrastructure perspective | 17:36 |
openstackgerrit | James E. Blair proposed a change to openstack-infra/zuul: Automatically provide 'name' to project templates https://review.openstack.org/62500 | 17:36 |
devananda | clarkb: yes, from an infra perspective too | 17:37 |
clarkb | devananda: you need a cloud with ironic running to do either tests | 17:37 |
devananda | clarkb: but in the currently-proposed tests, we don't need any VMs | 17:37 |
devananda | clarkb: it's just API exercises against a fake driver | 17:37 |
*** DennyZhang has joined #openstack-infra | 17:38 | |
clarkb | right but that fake driver still needs ironic to be running and to be configured... | 17:38 |
devananda | exercise our client, services, db, rpc, etc -- but not the drivers | 17:38 |
devananda | right | 17:38 |
devananda | and devstack can start ironic | 17:38 |
*** jcooley_ has quit IRC | 17:38 | |
*** senk has joined #openstack-infra | 17:38 | |
clarkb | enrollment of "baremetal" VMs seems to be the one difference | 17:38 |
devananda | the other stuff - -actually bringing up nodes with cirros on them, using ironic, will require changes to devstack-gate, a separate d-g pool, etc. | 17:38 |
devananda | clarkb: precisely | 17:39 |
romcheg | devananda, clarkb: All the changes to devstack and devstack-gate are merged already | 17:39 |
clarkb | I don't think that matters much from infra perspective, that would just be a step in the test that either runs or doesn't | 17:39 |
devananda | romcheg: no they're not :) | 17:39 |
romcheg | devananda: yes, they are :) | 17:39 |
*** sarob has quit IRC | 17:39 | |
romcheg | devananda: the one that's left is to infra config | 17:39 |
devananda | romcheg: the devstack-gate changes I'm referrign to (for ironic to actually start 100's of nodes with the pxe_ssh driver) aren't written yet. unless they are and I dont know about it | 17:40 |
clarkb | so anyways going back to where this discussion started, what is the concern? | 17:40 |
devananda | there are two TOTALLY DIFFERENT sets of tests taht I want: 1) fake driver 2) real driver. | 17:40 |
romcheg | devananda: Ah, I was talking about the API tests | 17:40 |
clarkb | devananda: right | 17:40 |
*** senk1 has quit IRC | 17:41 | |
devananda | all the things for testing ironic with FAKE driver are proposed, but it's been weeks and they haven't merged | 17:41 |
devananda | and there's been lots of back-and-forth about what test pipeline they should be in | 17:41 |
clarkb | where is the back and forth? | 17:41 |
devananda | https://review.openstack.org/#/c/48109/8 and https://review.openstack.org/#/c/53917/14 | 17:43 |
openstackgerrit | Anita Kuno proposed a change to openstack-infra/config: Turn on devstack exercises for neutron (non-voting) https://review.openstack.org/62930 | 17:43 |
*** MarkAtwood has joined #openstack-infra | 17:43 | |
clarkb | devananda: I don't think there was any back and forth in the infra change | 17:43 |
devananda | and on this thread http://lists.openstack.org/pipermail/openstack-dev/2013-December/021948.html | 17:43 |
clarkb | we just wanted the jobs to go in with the new templating | 17:43 |
*** jooools has quit IRC | 17:44 | |
jeblair | SergeyLukjanov: it looks like the alphabetized job isn't quite working: http://logs.openstack.org/17/53917/17/check/check-projects-yaml-alphabetized/71b08e3/console.html | 17:44 |
*** dkranz has joined #openstack-infra | 17:44 | |
clarkb | devananda: romcheg: can you guys tl;dr for me because I think I have read this stuff completely differently | 17:44 |
*** mkerrin has quit IRC | 17:44 | |
*** chandankumar has quit IRC | 17:45 | |
clarkb | devananda: romcheg: basically tempest wants to merge code that is actually tested upstream (which we are doing), and jobs can start in the experimental queue | 17:45 |
romcheg | devananda clarkb: the patch to tempest is actually waiting for the one to infra to test changes | 17:45 |
*** dolphm has quit IRC | 17:45 | |
clarkb | when the jobs are in the experimental queue you can `check experimental` on the tempest change and once it works you can switch to the check/gate queues | 17:45 |
romcheg | clarkb: I think devananda wants to be sure that the jobs will to to gate and check pipelines after they are checked in the experimental one | 17:46 |
flaper87 | fungi: fixed, thanks! :) https://review.openstack.org/#/c/32544/ | 17:53 |
*** flaper87 is now known as flaper87|afk | 17:56 | |
*** agordeev has quit IRC | 17:56 | |
*** amotoki has quit IRC | 17:56 | |
*** jpich has quit IRC | 17:56 | |
*** ArxCruz has quit IRC | 17:56 | |
*** oubiwann has quit IRC | 17:56 | |
*** thomasem has quit IRC | 17:56 | |
*** lcheng has quit IRC | 17:56 | |
*** lcestari has quit IRC | 17:56 | |
*** yamahata__ has quit IRC | 17:56 | |
*** romcheg has quit IRC | 17:56 | |
*** derekh has quit IRC | 17:56 | |
*** katyafervent has quit IRC | 17:56 | |
*** ianw has quit IRC | 17:56 | |
*** Hefeweizen has quit IRC | 17:56 | |
*** talluri_ has quit IRC | 17:56 | |
*** simonmcc has quit IRC | 17:56 | |
*** openstackgerrit has quit IRC | 17:56 | |
*** med_ has quit IRC | 17:56 | |
*** samalba has quit IRC | 17:56 | |
*** jeblair has quit IRC | 17:56 | |
*** jcooley has quit IRC | 17:56 | |
*** dragondm has quit IRC | 17:56 | |
*** DennyZhang has quit IRC | 17:56 | |
*** Abhishek has quit IRC | 17:56 | |
*** CaptTofu has quit IRC | 17:56 | |
*** sdake_ has quit IRC | 17:56 | |
*** mattymo has quit IRC | 17:56 | |
*** bknudson has quit IRC | 17:56 | |
*** briancline has quit IRC | 17:56 | |
*** mgagne has quit IRC | 17:56 | |
*** vishy has quit IRC | 17:56 | |
*** zigo has quit IRC | 17:56 | |
*** Adri2000 has quit IRC | 17:56 | |
*** cyeoh has quit IRC | 17:56 | |
*** persia has quit IRC | 17:56 | |
*** morganfainberg has quit IRC | 17:56 | |
*** vkozhukalov has quit IRC | 17:56 | |
*** pcrews has quit IRC | 17:56 | |
*** johnthetubaguy1 has quit IRC | 17:56 | |
*** spredzy has quit IRC | 17:56 | |
*** branen has quit IRC | 17:56 | |
*** vipul has quit IRC | 17:56 | |
*** HenryG has quit IRC | 17:56 | |
*** akscram has quit IRC | 17:56 | |
*** ilyashakhat has quit IRC | 17:56 | |
*** krtaylor has quit IRC | 17:56 | |
*** afazekas_ has quit IRC | 17:56 | |
*** dhellmann has quit IRC | 17:56 | |
*** rockstar has quit IRC | 17:56 | |
*** portante has quit IRC | 17:56 | |
*** jpeeler has quit IRC | 17:56 | |
*** thingee has quit IRC | 17:56 | |
*** basic` has quit IRC | 17:56 | |
*** pleia2 has quit IRC | 17:56 | |
*** SpamapS has quit IRC | 17:56 | |
*** odyi has quit IRC | 17:56 | |
*** hub_cap has quit IRC | 17:56 | |
*** koolhead17 has quit IRC | 17:56 | |
*** hemanth_ has quit IRC | 17:56 | |
*** kiall has quit IRC | 17:56 | |
*** BobBall has quit IRC | 17:56 | |
*** dkranz has quit IRC | 17:56 | |
*** yamahata has quit IRC | 17:56 | |
*** ryanpetrello has quit IRC | 17:56 | |
*** david-lyle_ has quit IRC | 17:56 | |
*** SergeyLukjanov has quit IRC | 17:56 | |
*** herndon has quit IRC | 17:56 | |
*** rnirmal has quit IRC | 17:56 | |
*** dizquierdo has quit IRC | 17:56 | |
*** rahmu has quit IRC | 17:56 | |
*** metabro_ has quit IRC | 17:56 | |
*** chmouel has quit IRC | 17:56 | |
*** Daviey has quit IRC | 17:56 | |
*** wayneeseguin has quit IRC | 17:56 | |
*** SlickNik has quit IRC | 17:56 | |
*** Mithrandir has quit IRC | 17:56 | |
*** antonym has quit IRC | 17:56 | |
*** sdague has quit IRC | 17:56 | |
*** EntropyWorks has quit IRC | 17:56 | |
*** mika has quit IRC | 17:56 | |
*** rainya has quit IRC | 17:56 | |
*** esmute has quit IRC | 17:56 | |
*** adam_g has quit IRC | 17:56 | |
*** stevebaker has quit IRC | 17:56 | |
*** ttx has quit IRC | 17:56 | |
*** kpepple has quit IRC | 17:56 | |
*** markmc has quit IRC | 17:56 | |
*** LinuxJedi has quit IRC | 17:56 | |
*** afazekas has quit IRC | 17:56 | |
*** niska has quit IRC | 17:57 | |
*** danger_f` has quit IRC | 17:57 | |
*** jog0 has quit IRC | 17:57 | |
*** harlowja_away has quit IRC | 17:57 | |
*** lifeless has quit IRC | 17:57 | |
*** dstufft has quit IRC | 17:57 | |
*** kashyap has quit IRC | 17:57 | |
*** tian has quit IRC | 17:57 | |
*** miqui has quit IRC | 17:57 | |
*** jswarren has quit IRC | 17:57 | |
*** jerryz has quit IRC | 17:57 | |
*** ywu has quit IRC | 17:57 | |
*** mfer has quit IRC | 17:57 | |
*** mriedem has quit IRC | 17:57 | |
*** rfolco has quit IRC | 17:57 | |
*** salv-orlando has quit IRC | 17:57 | |
*** hashar has quit IRC | 17:57 | |
*** boris-42 has quit IRC | 17:57 | |
*** shardy has quit IRC | 17:57 | |
*** sirushti has quit IRC | 17:57 | |
*** primeministerp has quit IRC | 17:57 | |
*** devananda has quit IRC | 17:57 | |
*** gema has quit IRC | 17:57 | |
*** krotscheck has quit IRC | 17:57 | |
*** rongze has quit IRC | 17:57 | |
*** dkehn has quit IRC | 17:57 | |
*** UtahDave has quit IRC | 17:57 | |
*** dims_ has quit IRC | 17:57 | |
*** julim has quit IRC | 17:57 | |
*** che-arne has quit IRC | 17:57 | |
*** StevenK has quit IRC | 17:57 | |
*** pabelanger has quit IRC | 17:57 | |
*** jd__ has quit IRC | 17:57 | |
*** ekarlso has quit IRC | 17:57 | |
*** Grishkin has quit IRC | 17:57 | |
*** milki has quit IRC | 17:57 | |
*** MarkAtwood has quit IRC | 17:57 | |
*** rwsu has quit IRC | 17:57 | |
*** sdake has quit IRC | 17:57 | |
*** dcramer_ has quit IRC | 17:57 | |
*** rcleere has quit IRC | 17:57 | |
*** esker has quit IRC | 17:57 | |
*** zul has quit IRC | 17:57 | |
*** weshay has quit IRC | 17:57 | |
*** freyes has quit IRC | 17:57 | |
*** mugsie has quit IRC | 17:57 | |
*** gaelL has quit IRC | 17:57 | |
*** juice has quit IRC | 17:57 | |
*** jheiss has quit IRC | 17:57 | |
*** vladan has quit IRC | 17:57 | |
*** uvirtbot has quit IRC | 17:57 | |
*** jesusaurus has quit IRC | 17:57 | |
*** bogdando has quit IRC | 17:57 | |
*** rpodolyaka has quit IRC | 17:57 | |
*** senk has quit IRC | 17:57 | |
*** ruhe has quit IRC | 17:57 | |
*** mrodden has quit IRC | 17:57 | |
*** dprince has quit IRC | 17:57 | |
*** crank has quit IRC | 17:57 | |
*** Shrews has quit IRC | 17:57 | |
*** jaypipes has quit IRC | 17:57 | |
*** mordred has quit IRC | 17:57 | |
*** dansmith has quit IRC | 17:57 | |
*** spiffxp_ has quit IRC | 17:57 | |
*** Loquacities has quit IRC | 17:57 | |
*** pvo has quit IRC | 17:57 | |
*** Guest33069 has quit IRC | 17:57 | |
*** AaronGr_Zzz has quit IRC | 17:57 | |
*** sileht has quit IRC | 17:57 | |
*** mancdaz_away has quit IRC | 17:57 | |
*** jlk has quit IRC | 17:57 | |
*** EmilienM has quit IRC | 17:57 | |
*** russellb has quit IRC | 17:57 | |
*** yongli has quit IRC | 17:57 | |
*** torgomatic has quit IRC | 17:57 | |
*** davidlenwell has quit IRC | 17:57 | |
*** iccha-dinner has quit IRC | 17:57 | |
*** hughsaunders has quit IRC | 17:57 | |
*** maurosr has quit IRC | 17:57 | |
*** saper has quit IRC | 17:57 | |
*** olaph has quit IRC | 17:57 | |
*** zaro has quit IRC | 17:57 | |
*** mindjiver has quit IRC | 17:57 | |
*** david-lyle has quit IRC | 17:57 | |
*** alaski has quit IRC | 17:57 | |
*** _cyril_ has quit IRC | 17:57 | |
*** guitarzan has quit IRC | 17:57 | |
*** GheRivero has quit IRC | 17:57 | |
*** hdd has quit IRC | 17:57 | |
*** clarkb has quit IRC | 17:57 | |
*** Ng has quit IRC | 17:57 | |
*** dachary has quit IRC | 17:57 | |
*** mikal has quit IRC | 17:57 | |
*** nikhil__ has quit IRC | 17:57 | |
*** westmau5 has quit IRC | 17:57 | |
*** jeremyb has quit IRC | 17:57 | |
*** greghaynes has quit IRC | 17:57 | |
*** Alex_Gaynor has quit IRC | 17:57 | |
*** fbo_away has quit IRC | 17:57 | |
*** notmyname has quit IRC | 17:57 | |
*** gfa has quit IRC | 17:57 | |
*** ewindisch has quit IRC | 17:57 | |
*** ogelbukh has quit IRC | 17:57 | |
*** adarazs has quit IRC | 17:57 | |
*** annegentle_ has quit IRC | 17:57 | |
*** enikanorov has quit IRC | 17:57 | |
*** mkoderer_ has quit IRC | 17:57 | |
*** belliott has quit IRC | 17:57 | |
*** nijaba has quit IRC | 17:57 | |
*** SushilKM has quit IRC | 17:57 | |
*** eharney has quit IRC | 17:57 | |
*** sandywalsh_ has quit IRC | 17:57 | |
*** slong has quit IRC | 17:57 | |
*** nicedice has quit IRC | 17:57 | |
*** bnemec has quit IRC | 17:57 | |
*** ChanServ has quit IRC | 17:57 | |
*** blamar has quit IRC | 17:57 | |
*** davidhadas has quit IRC | 17:57 | |
*** thomasbiege1 has quit IRC | 17:57 | |
*** yolanda has quit IRC | 17:57 | |
*** dkliban has quit IRC | 17:57 | |
*** Hunner has quit IRC | 17:57 | |
*** andreaf has quit IRC | 17:57 | |
*** adalbas has quit IRC | 17:57 | |
*** luisg has quit IRC | 17:57 | |
*** mtreinish has quit IRC | 17:57 | |
*** michchap has quit IRC | 17:57 | |
*** fungi has quit IRC | 17:57 | |
*** sbadia has quit IRC | 17:57 | |
*** anteaya has quit IRC | 17:57 | |
*** openstack has joined #openstack-infra | 18:02 | |
*** openstack has joined #openstack-infra | 18:06 | |
ruhe | krotscheck: jfyi there is a #storyboard channel | 18:06 |
hub_cap | fungi: fair enough | 18:06 |
*** herndon has quit IRC | 18:07 | |
krotscheck | ruhe: Huhn- I thought that'd be called #openstack-storybaord | 18:07 |
SergeyLukjanov | fungi, anteaya, thanks for catching it, I'll fix it when I'll be home ;) | 18:07 |
fungi | SergeyLukjanov: sounds like anteaya may beat you to it anyway | 18:08 |
*** dcramer_ has joined #openstack-infra | 18:08 | |
fungi | krotscheck: i think the goal is to make storyboard generally reusable beyond the openstack community, thus it's not an openstack-specific project | 18:08 |
* anteaya is checking to see if she is working on the correct file | 18:08 | |
anteaya | SergeyLukjanov: config/infra.yaml? | 18:09 |
fungi | anteaya: the script being run in that log | 18:09 |
*** krotscheck has left #openstack-infra | 18:09 | |
*** krotscheck has joined #openstack-infra | 18:09 | |
krotscheck | fungi: So there's a naming convention there, got it. | 18:09 |
*** harlowja_away is now known as harlowja | 18:09 | |
*** dstanek has quit IRC | 18:10 | |
anteaya | fungi: ah | 18:10 |
* anteaya looks again | 18:10 | |
clarkb | devananda: romcheg: I am guessing that clears up the confusion? I hope to catch up on reviews today between running errands. will rereview the infra change | 18:10 |
*** herndon has joined #openstack-infra | 18:11 | |
fungi | anteaya: http://git.openstack.org/cgit/openstack-infra/config/tree/modules/jenkins/files/slave_scripts/check_projects_yaml_alphabetized.sh#n12 | 18:11 |
romcheg | clarkb: For testing api is clear. | 18:11 |
anteaya | fungi: thanks, just found it | 18:11 |
fungi | anteaya: make it $OLDPWD/modules/openstack_project/templates/review.projects.yaml.erb on that line | 18:12 |
anteaya | will do | 18:12 |
romcheg | clarkb: Maybe you can help me with that DEVSTACK_GATE_SMOKE_SERIAL flag? | 18:12 |
*** gyee has joined #openstack-infra | 18:12 | |
*** psedlak has joined #openstack-infra | 18:13 | |
*** mestery has joined #openstack-infra | 18:13 | |
*** saschpe has joined #openstack-infra | 18:13 | |
*** beagles has joined #openstack-infra | 18:13 | |
*** dripton_ has joined #openstack-infra | 18:13 | |
*** locke105 has joined #openstack-infra | 18:13 | |
*** changbl has joined #openstack-infra | 18:13 | |
*** flaper87|afk has joined #openstack-infra | 18:13 | |
*** NobodyCam has joined #openstack-infra | 18:13 | |
*** openstackstatus has joined #openstack-infra | 18:13 | |
*** soren has joined #openstack-infra | 18:13 | |
*** dtroyer_zz has joined #openstack-infra | 18:13 | |
romcheg | clarkb: https://review.openstack.org/#/c/53917/16/modules/openstack_project/files/jenkins_job_builder/config/devstack-gate.yaml | 18:13 |
romcheg | clarkb: the bug seems to be fixed but several neutron jobs still set the flag. | 18:13 |
*** SushilKM has quit IRC | 18:13 | |
clarkb | romcheg: I don't think neutron has fixed it. If that have they have not update their jobs to actually gate on it... | 18:15 |
*** Alex_Gaynor has quit IRC | 18:15 | |
openstackgerrit | A change was merged to openstack-infra/config: Add status.openstack.org vm https://review.openstack.org/61552 | 18:15 |
clarkb | romcheg: the jobs with neutron-isolated-parallel in their name do run in parallel | 18:15 |
clarkb | you can check if those are passing | 18:15 |
*** BobBall is now known as BobBallAway | 18:15 | |
*** Abhishek has quit IRC | 18:15 | |
lifeless | sdague: o/ | 18:16 |
clarkb | romcheg: they are in the experimental queue for tempest and neutron iirc, so you check check experimental on your tempest api test change to see if they pass | 18:17 |
*** Abhishek has joined #openstack-infra | 18:17 | |
*** Alex_Gaynor_ has joined #openstack-infra | 18:17 | |
openstackgerrit | Joe Gordon proposed a change to openstack-infra/elastic-recheck: Add query for bug 1262154 https://review.openstack.org/62948 | 18:18 |
uvirtbot | Launchpad bug 1262154 in nova "HTTP 500 on update_server_metadata (DB deadlock)" [Undecided,Confirmed] https://launchpad.net/bugs/1262154 | 18:18 |
anteaya | fungi: I didn't get my prior patch up before changing to a new branch for the script change, now I have them all folded into one commit | 18:18 |
anteaya | fungi: looking at git reset, should I be looking elsewhere? | 18:18 |
fungi | anteaya: might be easiest to just undo your trivial edit to that one file and git commit -a --amend again, then switch to a new branch and reapply that edit | 18:19 |
anteaya | I'll do that | 18:19 |
*** jcooley_ has joined #openstack-infra | 18:20 | |
*** jcooley_ has quit IRC | 18:21 | |
fungi | if it were tougher to undo, i'd suggest you look at resetting to a sha from your reflog | 18:21 |
anteaya | fungi: yeah, let's wait on that dance | 18:22 |
jeblair | fungi, clarkb: i'm stopping puppet on the logstash workers and client and applying changes | 18:22 |
*** nati_ueno has joined #openstack-infra | 18:22 | |
fungi | jeblair: sounds good | 18:22 |
*** jgriffith has joined #openstack-infra | 18:23 | |
openstackgerrit | A change was merged to openstack-infra/config: Refactor logstash_worker into log_processor module https://review.openstack.org/62787 | 18:27 |
openstackgerrit | Anita Kuno proposed a change to openstack-infra/config: Fix the yaml-alphabetized job https://review.openstack.org/62951 | 18:27 |
openstackgerrit | A change was merged to openstack-infra/config: Increase to 16 logstash worker nodes https://review.openstack.org/62788 | 18:29 |
jlk | mordred: hey there, got another favor to ask wrt rackspace-auth-openstack. Could you ask github to make the rackerlabs/rackspace-auth-openstack fork the new root of "rackspace-auth-openstack" ? | 18:30 |
jlk | mordred: since you own the current root github says you gotta ask for it. | 18:30 |
*** johnthetubaguy1 has quit IRC | 18:30 | |
reed | fungi, jeblair: https://review.openstack.org/#/c/61250/ time to merge it? | 18:32 |
fungi | clearly freenode is under orbital bombardment again... i keep getting timeouts to my current irc server | 18:33 |
*** dkranz has quit IRC | 18:35 | |
*** dstufft has quit IRC | 18:35 | |
*** dstufft has joined #openstack-infra | 18:36 | |
*** Alex_Gaynor_ is now known as Alex_Gaynor | 18:37 | |
*** Alex_Gaynor has quit IRC | 18:37 | |
*** Alex_Gaynor has joined #openstack-infra | 18:37 | |
*** jpich has quit IRC | 18:38 | |
*** dstufft has joined #openstack-infra | 18:38 | |
*** ruhe has quit IRC | 18:40 | |
* sdague catching up on scrollback | 18:42 | |
sdague | romcheg: as far as I know neutron still won't work parallel | 18:43 |
*** blamar has quit IRC | 18:43 | |
sdague | if someone think that's fixed, we should revisit | 18:43 |
anteaya | sdague: as far as I know, not working parallel yet | 18:44 |
anteaya | salv-orlando is working on it | 18:44 |
*** markmcclain has joined #openstack-infra | 18:45 | |
romcheg | sdague: Thank you | 18:45 |
anteaya | https://blueprints.launchpad.net/neutron/+spec/neutron-tempest-parallel | 18:45 |
salv-orlando | current trunk code has perhaps at beast a 15% success rate on parallel job. We have already patches under review which *hopefully* should bring that to 80% at least. | 18:46 |
jeblair | fungi, clarkb, mordred: you okay with canceling the next two infra meetings? they are on christmas eve day and new years eve day respectively. | 18:47 |
clarkb | jeblair: wfm | 18:47 |
clarkb | I will probably be around if we decide there is areason to have those meetings | 18:47 |
fungi | jeblair: i'll chair those, in an overstuffed arm chair, with my special eggnog which is made by pouring burbon in a glass | 18:48 |
*** markmc has quit IRC | 18:49 | |
fungi | but yeah, cancelling is fine with me | 18:49 |
jeblair | fungi: that's not how you normally chair them? | 18:49 |
*** dkranz has joined #openstack-infra | 18:49 | |
*** salv-orlando_ has joined #openstack-infra | 18:49 | |
fungi | good point | 18:49 |
*** salv-orlando has quit IRC | 18:50 | |
clarkb | fungi: you need a monocle as well | 18:50 |
*** salv-orlando_ is now known as salv-orlando | 18:50 | |
fungi | and a calabash | 18:50 |
anteaya | salv-orlando: thanks for the update | 18:50 |
*** SushilKM has joined #openstack-infra | 18:50 | |
* anteaya is looking forward to the next two infra meetings | 18:51 | |
jeblair | launching worker01-08 nodes (4g performance) | 18:51 |
fungi | reed: i think https://review.openstack.org/61250 is ready to merge, just want to get clarkb or jeblair to skim the final patchset in case they have any other comments | 18:51 |
* fungi would rather avoid self-approving a new infra project addition, mostly on principle | 18:51 | |
openstackgerrit | Michael Krotscheck proposed a change to openstack-infra/storyboard: Added .gitignore for IntelliJ specific configuration files https://review.openstack.org/62956 | 18:52 |
*** kgriffs has joined #openstack-infra | 18:52 | |
jeblair | fungi, reed: lgtm | 18:52 |
clarkb | fungi: if you approve with a glass of bourbon while wearing a monocle in a large overstuffed chair I don't think we can complain | 18:52 |
clarkb | reviewing as well | 18:53 |
fungi | heh | 18:53 |
krotscheck | Woo, first CR | 18:53 |
fungi | zaro: the review-dev rollback worked as far as i can tell. i imported the musqldump from the day before the upgrade while gerrit was stopped, then reverted 7011082 in a current puppet development environment and applied | 18:53 |
fungi | krotscheck: welcome aboard officially now! | 18:54 |
krotscheck | fungi: Thanks! | 18:54 |
krotscheck | Except that jenkings barfed. | 18:54 |
sdague | lifeless: what's up? | 18:54 |
lifeless | sdague: you invoked my name | 18:54 |
pleia2 | anteaya: did you catch that the next two meetings are next year? (not doing xmas eve and nye) | 18:54 |
lifeless | sdague: in the context of testr/nose | 18:54 |
fungi | krotscheck: you will get used to that. jenkins hates us and our attempts to write clean code | 18:54 |
krotscheck | fungi: ...yeah, in this case it really seems to hate gitignore. | 18:55 |
openstackgerrit | A change was merged to openstack-infra/config: Import the openstackid authentication project https://review.openstack.org/61250 | 18:55 |
fungi | krotscheck: it looks to me like storyboard.stories.tests.SimpleTest.test_even_more_addtion may have bitrotted or something | 18:56 |
openstackgerrit | James E. Blair proposed a change to openstack-infra/config: Support launching rax performance nodes https://review.openstack.org/62958 | 18:56 |
jeblair | fungi, clarkb: ^ just tested in production | 18:56 |
fungi | krotscheck: or that job has been perpetually busted, or doesn't clean up properly before running or something | 18:56 |
fungi | jeblair: that's where i test all my code | 18:57 |
krotscheck | fungi: Thanks for pointing me in the right direction, I'm still new to python :) | 18:57 |
openstackgerrit | James E. Blair proposed a change to openstack-infra/config: Support launching rax performance nodes https://review.openstack.org/62958 | 18:57 |
krotscheck | fungi: Awwww yeah. Nothing like a helloworld test case breaking your jenkins. | 18:58 |
*** afazekas_ has quit IRC | 18:59 | |
fungi | krotscheck: given the project is still in its infancy, i wouldn't be surprised if the voting unit test job was added even though the unit tests didn't run | 18:59 |
anteaya | pleia2: my tongue in cheek response to saying that fungi chairing with eyepiece and beverage should be a good time | 18:59 |
anteaya | pleia2: and yes, got that they are cancelled | 19:00 |
*** dstanek has joined #openstack-infra | 19:00 | |
sdague | lifeless: right, I want to remove nose support from tempest with a vengence | 19:00 |
lifeless | what do you need from testr to do that? | 19:00 |
anteaya | pleia2: thanks for making sure I am up to date, now that I am rarely here | 19:00 |
sdague | but I remember there being some weird condition on 2.6 + testr that is still not resolved | 19:00 |
anteaya | :D | 19:00 |
clarkb | lifeless: sdague: the weird case is skip test exceptions | 19:00 |
fungi | anteaya: i can take the meeting as an opportunity for story time. maybe a dramatic rendition of 20-year-old bofh newsletters or something | 19:00 |
pleia2 | anteaya: oh ok :) | 19:00 |
clarkb | nose adds them itself in 2.6 and making that work under 2.6 is weird apparently. also setup class but unittest2 can apparently handle that | 19:01 |
lifeless | right, i spent some time with afazekas on that at the summit | 19:01 |
lifeless | bugs everywhere | 19:01 |
krotscheck | fungi: Makes sense. Also def. the right decision to make. | 19:01 |
sdague | ok, any idea how close it got with afazekas ? | 19:01 |
*** jswarren has quit IRC | 19:01 | |
anteaya | fungi krotscheck they were | 19:02 |
anteaya | fungi: I'm looking forward to storytime | 19:02 |
fungi | anteaya: krotscheck: so if there's a pending change in review to fix the unit tests, you can check that out and rebase your .gitignore change on top of it | 19:02 |
*** sarob has joined #openstack-infra | 19:02 | |
fungi | and then they should be tested together and pass | 19:03 |
anteaya | turns out a django project must have at least 4 tests otherwise jenkins gets upset | 19:03 |
clarkb | anteaya: we fixed that | 19:03 |
anteaya | ah, see so long ago | 19:03 |
clarkb | anteaya: now you need 1. running 0 tests is a fail | 19:03 |
anteaya | ha ha ha | 19:04 |
fungi | in the before-time, in the long-long-ago. like, days ago | 19:04 |
anteaya | where was that changed, in the jenkins job script? | 19:04 |
anteaya | fungi: yeah, that time frame | 19:04 |
openstackgerrit | A change was merged to openstack-infra/elastic-recheck: Add query for bug 1262154 https://review.openstack.org/62948 | 19:05 |
uvirtbot | Launchpad bug 1262154 in nova "HTTP 500 on update_server_metadata (DB deadlock)" [Medium,In progress] https://launchpad.net/bugs/1262154 | 19:05 |
fungi | anteaya: modules/jenkins/files/slave_scripts/run-unittests.sh 8c28621b (Clark Boylan 2013-12-06 12:57:07 -0800) | 19:06 |
fungi | so very, very long ago | 19:06 |
krotscheck | fungi: Looks like lifeless & nikita have an initial test set, I'll rebase on top of that. | 19:07 |
*** dstufft_ has joined #openstack-infra | 19:07 | |
fungi | krotscheck: if it's shown as passing the defined jenkins jobs, then that ought to be what you want | 19:07 |
*** Abhishek has quit IRC | 19:07 | |
anteaya | ages | 19:08 |
*** dkranz has quit IRC | 19:08 | |
fungi | granted, things move so fast here that a week seems like a year sometimes | 19:08 |
*** jergerber has joined #openstack-infra | 19:08 | |
sdague | nice... try to submit feedback on https://www.openstack.org/foundation/technical-committee/ with the form at the bottom of the page | 19:09 |
reed | fungi, jeblair, woot! thanks! | 19:09 |
*** davidhadas has quit IRC | 19:09 | |
sdague | it thinks it a cross site scripting attack | 19:09 |
*** dstufft has quit IRC | 19:10 | |
openstackgerrit | Clark Boylan proposed a change to openstack-infra/config: Publish elastic-recheck documentation. https://review.openstack.org/61300 | 19:10 |
clarkb | sdague: jeblair ^ that should publish it to the desired location | 19:10 |
reed | sdague, who thinks it is a xss? | 19:10 |
fungi | sdague: are you sure you're not trying to pwn us all? (but in seriousness we get on the order of a report a week for some vulnerability in that site, so tipit have been putting in a lot of paranoia checks to deal with those) | 19:11 |
sdague | reed the site I think | 19:12 |
*** dstufft_ has quit IRC | 19:12 | |
sdague | oh, no, chrome | 19:12 |
sdague | "Security token doesn't match, possible CSRF attack." | 19:12 |
fungi | ahh, neat | 19:12 |
sdague | honestly, no idea really, but again with the broken. | 19:13 |
*** dstufft has joined #openstack-infra | 19:13 | |
*** shivh has joined #openstack-infra | 19:13 | |
fungi | sdague: you probably just want to file something at https://bugs.launchpad.net/openstack-org | 19:14 |
clarkb | fungi: https://review.openstack.org/#/admin/projects/openstack-infra/openstackid,access looks like that still isn't working quite right. github was created and code exists there. the openstackid-core group was created but not the openstackid-ptl group which I am guessing caused the acl push to fail | 19:15 |
clarkb | I have no idea why that is happening. If one group creation works the other should as well in theory. I mean the code itself isn't broken right? | 19:16 |
jeblair | err: Could not prefetch package provider 'pip': | 19:16 |
jeblair | clarkb: ^ have you seen that puppet error ? | 19:16 |
fungi | clarkb: yep, last time i got lucky and encountered it manually, the error was "ERROR:manage_projects:Unable to get UUID for group gantt-milestone." | 19:16 |
shivh | clarkb: I am experimenting with tempest tests posting verify (third party testing) and have some problem with posting "verified=1", however i dont have a problem posting "verifyed=0". Anyone out there know why. THanks. | 19:17 |
clarkb | fungi: right so I think it must be a race around creating the group and it being in the tables? | 19:17 |
fungi | clarkb: then re-ran it and it worked fine | 19:17 |
fungi | clarkb: that's my current suspicion | 19:17 |
clarkb | jeblair: the pip provider is something that will be copied to the host before puppet runs the catalog | 19:17 |
jeblair | shivh: you're probably not using an account registered with gerrit; see http://ci.openstack.org/third_party.html | 19:17 |
clarkb | jeblair: I think it is saying that that initial copy failed for some reason. I don't recall running into that particular error but ^ is what I believe is happening | 19:17 |
jeblair | shivh: rather, it's not registered as a third party test system | 19:17 |
jeblair | clarkb: retry doesn't help | 19:18 |
clarkb | fungi: maybe we put the busy wait around the db querying | 19:18 |
jeblair | clarkb: ++ (busy wait) | 19:18 |
*** dstanek has quit IRC | 19:19 | |
jeblair | shivh: see http://ci.openstack.org/third_party.html#posting-result-to-gerrit and http://ci.openstack.org/third_party.html#requesting-a-service-account | 19:19 |
fungi | clarkb: sounds like an excellent idea | 19:19 |
*** dkranz has joined #openstack-infra | 19:20 | |
clarkb | jeblair: do you have accesss to the host? is pip installed? I wonder if maybe the new pip stuff mordred did depends on having pip ahead of time? | 19:20 |
jeblair | clarkb: http://paste.openstack.org/show/55365/ | 19:20 |
jeblair | hrm. | 19:20 |
clarkb | jeblair: ok I bet it is related to broken pip install | 19:21 |
clarkb | jeblair: that looks like classic case of need to uninstall pip and pkg-resources, reinstall pip then immediately update setuptools | 19:22 |
*** dstanek has joined #openstack-infra | 19:23 | |
*** mrodden has quit IRC | 19:26 | |
anteaya | clarkb jeblair pleia2 krotscheck AaronGr_Zzz can I get some reviews? I'd like to get this in so I can get another patch in: https://review.openstack.org/#/c/62951/ | 19:26 |
anteaya | and zaro | 19:27 |
openstackgerrit | Russell Bryant proposed a change to openstack-infra/config: Make gate-solum-devstack-dsvm voting https://review.openstack.org/62964 | 19:27 |
jeblair | anteaya: please be patient; a few hours is not a long time to wait for reviews; expect a few days. and to continue working, make your new patchset depend on that one. | 19:27 |
anteaya | jeblair: it isn't, I can get the other in without this test passing, would just be nice if it was | 19:28 |
anteaya | sorry to bother you | 19:28 |
openstackgerrit | Michael Krotscheck proposed a change to openstack-infra/storyboard: Added .gitignore for IntelliJ specific configuration files https://review.openstack.org/62956 | 19:29 |
anteaya | fungi: so if the turning on of devstack exercises gets in, I can have jlibovar keep working on it tomorrow | 19:29 |
jeblair | clarkb: do you know how the system got into this state? | 19:30 |
jeblair | clarkb: during the initial run, it encountered this error: http://paste.openstack.org/show/55367/ | 19:32 |
*** CaptTofu has quit IRC | 19:32 | |
jeblair | clarkb: it seems to be able to run pip at that time, and i don't see any pip-related actions after it | 19:32 |
*** CaptTofu has joined #openstack-infra | 19:33 | |
clarkb | jeblair: could it be a bad interaction between old pip and gear+pbr? | 19:34 |
*** mrodden has joined #openstack-infra | 19:34 | |
clarkb | jeblair: also note that that is /usr/lib/python*/pip not /usr/local/lib... I think puppet is upgrading pip from 1.0 to 1.4.1 with pip. That could also possibly be at fault | 19:35 |
*** dkranz has quit IRC | 19:36 | |
jeblair | clarkb: where does pip get upgraded? | 19:36 |
jeblair | clarkb: indeed, there seems to be no /usr/bin/pip anymore, despite the debian package being installed. | 19:37 |
openstackgerrit | Clark Boylan proposed a change to openstack-infra/jeepyb: Wait for DB to reflect group UUID updates https://review.openstack.org/62967 | 19:37 |
clarkb | jeblair: in the pip module iirc | 19:37 |
jeblair | clarkb: i'm not seeing it; it looks like it should just install python-pip from the os | 19:38 |
clarkb | hmm nevermind that doesn't seem to be it | 19:38 |
clarkb | ya | 19:38 |
jeblair | clarkb: which is what i see in the logs too | 19:38 |
fungi | i think the intent was for the distro package of pip to get uninstalled, then get-pip retrieves and installs it from github/pypi | 19:38 |
clarkb | jeblair: pbr requires new pip perhaps it is doing it for us? | 19:38 |
*** herndon has left #openstack-infra | 19:38 | |
jeblair | fungi: i don't see any code to do that :/ | 19:39 |
openstackgerrit | Russell Bryant proposed a change to openstack-infra/config: Make gate-solum-devstack-dsvm voting https://review.openstack.org/62964 | 19:39 |
fungi | oh, but mordred's pip provider refactor rotted on the vine i think, which is where that was happening | 19:39 |
jeblair | clarkb: possibly so. | 19:39 |
fungi | it was broken for python 3.3 pip i believe when i tested it, and i'm not sure it got a new patchset after that | 19:39 |
*** krotscheck has quit IRC | 19:40 | |
jeblair | so i believe we do not have the ability to launch a new server that runs a python package that uses pbr. | 19:41 |
jeblair | which is quite a few of our servers. | 19:41 |
mordred | anything I can look at to be helpful? | 19:42 |
openstackgerrit | A change was merged to openstack-infra/devstack-gate: Don't fix disks before rexecing https://review.openstack.org/60961 | 19:43 |
jeblair | mordred: yes. i have no idea how to fix that. fungi says you were working on it. perhaps you could indicate what is needed? | 19:43 |
fungi | mordred: i think https://review.openstack.org/51425 needs restoring and continuing | 19:43 |
mordred | well - it's a really hard problem which ultimately always makes us want to move forward with pabelanger's packaging work of our stuff | 19:43 |
jeblair | mordred: a brand new spinup of a server gets this error: http://paste.openstack.org/show/55367/ and then pip is broken: http://paste.openstack.org/show/55365/ | 19:44 |
jeblair | mordred: i do seem to recall that being the case. i didn't think we were quite so broken at the moment though. | 19:44 |
mordred | jeblair: I didn't either | 19:44 |
*** dcramer_ has quit IRC | 19:45 | |
mordred | jeblair: it's fixable by hand - the above patch is the one that should fix it systemically | 19:45 |
mordred | jeblair: I have an idea for a short-term fix | 19:46 |
jeblair | mordred: it looks like that patch is not fit to land atm, so how do i fix it by hand? | 19:46 |
jeblair | mordred: i'll need to do that on the 16 machines i'm spinning up | 19:46 |
mordred | jeblair: why don't we add the get-pip invocation to launch-node.py | 19:46 |
mordred | so that we pre-get pip installed properly on the machine before we try to run puppet | 19:46 |
mordred | which will get you unstuck | 19:46 |
mordred | it's not ideal, obviously | 19:47 |
jeblair | mordred: i think it should be added to install_puppet.sh so it works for people who don't use launch-node | 19:48 |
mordred | jeblair: tools/install_pip.sh in devstack describes how pip wants to get installed | 19:48 |
mordred | jeblair: ++ | 19:48 |
mordred | jeblair: lemme make a patch real quick | 19:48 |
jd__ | mordred: can I stare at you and frown because you never answered https://review.openstack.org/#/c/58770/ ? | 19:48 |
mordred | jd__: yeah, but lemme get this done for jeblair first | 19:49 |
jeblair | mordred: i think puppet wants python-pip package installed, but tools/install_pip wants it removed; so maybe your patch should also remove that? | 19:49 |
mordred | I haven't been very useful recently | 19:49 |
mordred | jeblair: yes. I believe you are correct | 19:49 |
jd__ | mordred: sure! ;-) | 19:50 |
* jd__ stares at jeblair | 19:50 | |
jeblair | mordred: https://etherpad.openstack.org/p/RHbv5NLdi0 | 19:51 |
*** dkranz has joined #openstack-infra | 19:51 | |
shivh | clarkb: I should be "registered with gerrit". I have been contributing code etc. looks like something subtle i don't understand. | 19:52 |
*** jcooley_ has joined #openstack-infra | 19:53 | |
anteaya | jd__: don't do that, you will get a slap on the wrist | 19:53 |
anteaya | give him his work space | 19:53 |
jd__ | I can stare from a very far place to avoid that :-) | 19:54 |
anteaya | ha ha ha | 19:55 |
anteaya | jim has long arms | 19:55 |
*** davidhadas has joined #openstack-infra | 19:56 | |
fungi | shivh: the point was that to leave a verify vote you need to request a specific gerrit service account as described in the links you were provided earlier. it gets added to a special group which has verify voting ability | 19:56 |
fungi | shivh: normal gerrit user accounts are not allowed to add verify votes | 19:56 |
fungi | (in openstack's configuration at any rate) | 19:57 |
shivh | thanks. got it. | 19:57 |
*** dizquierdo has quit IRC | 19:57 | |
shivh | fungi, clarkb, jeblair: Thanks. | 19:57 |
*** jcooley_ has quit IRC | 19:57 | |
*** dcramer_ has joined #openstack-infra | 19:58 | |
fungi | flaper87|afk: looks like you logged off that test host. are you finished investigating the failure there? if so i'll discard it | 20:01 |
jeblair | mordred: etherpad now works; the python-pip package seems to be mostly irrelevant | 20:02 |
openstackgerrit | Jeremy Stanley proposed a change to openstack-infra/config: Support launching rax performance nodes https://review.openstack.org/62958 | 20:03 |
openstackgerrit | Monty Taylor proposed a change to openstack-infra/config: Install pip using get-pip in launch node https://review.openstack.org/62973 | 20:04 |
mordred | jeblair: ^^ | 20:04 |
mordred | jeblair: the larger patch I thnkn still wants to be fixed, btw | 20:04 |
mordred | and we shoudl talk about python2 and python3 colocated on the same host with the patch | 20:05 |
jeblair | cool. i thought that wasn't going to work? | 20:05 |
mordred | it won't | 20:05 |
*** shivh has quit IRC | 20:05 | |
fungi | i'm rather amused that this mechanism for installing pip requires access to bitbucket, github and pypi | 20:06 |
jeblair | fungi: what could possibly go wrong? | 20:06 |
mordred | but I think there was concern with how python3-pip installed a pip3 thing | 20:06 |
*** david-lyle_ has quit IRC | 20:06 | |
*** dkranz has quit IRC | 20:07 | |
fungi | mordred: particularly, that the module search path for colocated python2.7 and python3.3 interpreters on debian-derived distros is different, and calling the pip3 vs pip entrypoint is how pip knows where to install packages so that they'll be found by the right interpreter | 20:08 |
fungi | i wanted to test out the rewritten provider to see whether it would "just work" anyway, but currently puppet doesn't like it (error messages linked from my last comment in the old review) | 20:09 |
*** davidhadas has quit IRC | 20:10 | |
*** sandywalsh_ has quit IRC | 20:12 | |
fungi | but the crux of the issue is that if you python2.7 -c 'import sys;print(sys.path)' and then do the same for 3.3 and compare them, you'll see that the only entry they share is the current working directory | 20:12 |
fungi | all others are qualified by the interpreter version | 20:12 |
fungi | (and none seem to be symlinked to one another that i could find) | 20:13 |
*** derekh has joined #openstack-infra | 20:13 | |
* clarkb is going to afk for lunch and errands now. back later | 20:14 | |
mordred | fungi: oh! I think I know how to fix it | 20:19 |
fungi | awesome! | 20:19 |
*** dkranz has joined #openstack-infra | 20:21 | |
openstackgerrit | A change was merged to openstack-infra/config: Support launching rax performance nodes https://review.openstack.org/62958 | 20:21 |
*** prad has joined #openstack-infra | 20:22 | |
*** praneshp has joined #openstack-infra | 20:22 | |
mordred | fungi: if we make pip.rb call "python -m pip.main" instead of "pip" - and the other one "python3 -m pip.main" | 20:22 |
mordred | then we're set | 20:22 |
*** elasticio has joined #openstack-infra | 20:23 | |
*** sandywalsh_ has joined #openstack-infra | 20:24 | |
jeblair | clarkb, fungi, mordred: logstash worker01 A and B are running now | 20:24 |
mordred | jeblair: woot | 20:25 |
jeblair | one oversight: they both report the same worker name to gearman; but that's not that important and we can fix later. | 20:26 |
fungi | mordred: sounds reasonable | 20:27 |
*** vkozhukalov has quit IRC | 20:28 | |
*** yolanda has quit IRC | 20:33 | |
*** _david_ has joined #openstack-infra | 20:35 | |
openstackgerrit | James E. Blair proposed a change to openstack-infra/devstack-gate: Remove strange duplication of setup_host and output https://review.openstack.org/62983 | 20:36 |
openstackgerrit | Darragh Bailey proposed a change to openstack-infra/jenkins-job-builder: Use yaml local tags to support including files https://review.openstack.org/48783 | 20:36 |
openstackgerrit | A change was merged to openstack-infra/config: Install pip using get-pip in launch node https://review.openstack.org/62973 | 20:36 |
openstackgerrit | Roman Prykhodchenko proposed a change to openstack-infra/config: Add devstack-gate jobs for Ironic https://review.openstack.org/53917 | 20:37 |
*** sdake_ is now known as sdake2 | 20:38 | |
*** sarob has quit IRC | 20:40 | |
*** sarob has joined #openstack-infra | 20:40 | |
jeblair | clarkb, mordred, fungi: all of the old log workers are stopped, and the first 8 of the new hosts (16 workers) are running | 20:41 |
*** AaronGr_Zzz is now known as AaronGr | 20:41 | |
*** hashar has joined #openstack-infra | 20:42 | |
AaronGr | back now, hi folks. | 20:42 |
fungi | jeblair: i guess we didn't add them to cacti yet | 20:42 |
fungi | AaronGr: welcome back! | 20:42 |
fungi | jeblair: i'll get the cacti addition whipped up | 20:43 |
mordred | hey it's AaronGr | 20:43 |
jeblair | fungi: cool, thanks | 20:44 |
AaronGr | mordred: just made it home, going to get email and vpn working, then get back to gerrit and ldap. | 20:44 |
*** derekh has quit IRC | 20:45 | |
fungi | AaronGr: protip, you don't need that hp vpn or e-mail. i haven't looked at mine in months | 20:45 |
fungi | oh, wait | 20:45 |
AaronGr | fungi: having a few conversations at the moment where that's the endpoint.. and all the boxes i need to hit today/tomorrow are behind a vpn | 20:46 |
fungi | AaronGr: don't listen to me--i haven't worked for hp since last spring ;) | 20:46 |
*** dcramer_ has quit IRC | 20:47 | |
AaronGr | fungi: no worries | 20:47 |
*** smarcet has left #openstack-infra | 20:50 | |
*** dprince has quit IRC | 20:50 | |
*** senk has quit IRC | 20:53 | |
openstackgerrit | Andreas Jaeger proposed a change to openstack-infra/config: Further setup of openstack-doc-tools https://review.openstack.org/62984 | 20:54 |
*** mrodden has quit IRC | 20:54 | |
*** mrodden has joined #openstack-infra | 20:56 | |
*** marun has joined #openstack-infra | 20:56 | |
openstackgerrit | Jeremy Stanley proposed a change to openstack-infra/config: Add performance flavor logstash workers to cacti https://review.openstack.org/62985 | 20:56 |
fungi | this way ^ we can trend how loaded they get and see how many we really need | 20:57 |
*** sarob has quit IRC | 20:58 | |
*** hogepodge has joined #openstack-infra | 21:00 | |
*** krotscheck has joined #openstack-infra | 21:02 | |
*** sarob has joined #openstack-infra | 21:02 | |
*** krotscheck has quit IRC | 21:08 | |
openstackgerrit | A change was merged to openstack-infra/config: Add performance flavor logstash workers to cacti https://review.openstack.org/62985 | 21:11 |
*** mancdaz_away is now known as mancdaz | 21:12 | |
*** SergeyLukjanov has quit IRC | 21:12 | |
*** miqui has quit IRC | 21:14 | |
*** fbo_away is now known as fbo | 21:21 | |
gyee | sdague, I am trying to chase down a grenade upgrade failure for https://review.openstack.org/#/c/56333/ | 21:23 |
gyee | the log seem to suggest that keystone-paste.ini is not upgraded because I don't see the new filter being called | 21:24 |
fungi | gyee: grenade tests new openstack with old configuration, so it won't be | 21:24 |
gyee | ah | 21:24 |
fungi | gyee: we want to make sure that deployers can upgrade from havana to icehouse and still have a working openstack *before* they need to go making configuration adjustments | 21:25 |
fungi | gyee: that makes things like rolling upgrades more reasonable to accomplish in production | 21:25 |
gyee | fungi, so its running IceHouse with Havana config? | 21:25 |
sdague | yes | 21:25 |
fungi | gyee: yes | 21:25 |
gyee | k, let me make the adjustment then | 21:26 |
gyee | fungi, sdague, thanks | 21:26 |
fungi | needs to degrade gracefully using reasonable built-in defaults, basically | 21:26 |
sdague | if we really can't get around that we can flag it as UpgradeImpact and provide a work around, but we are really trying to basically make the work arounds noop | 21:27 |
*** dcramer_ has joined #openstack-infra | 21:27 | |
gyee | btw, I did tried to run grenade on my local VM, not sure if I was following the steps correctly | 21:27 |
gyee | I basically checkout devstack and grenade, ran setup-grenade localhost | 21:28 |
gyee | than modify grenaderc to use my review branch | 21:28 |
gyee | then grenade.sh | 21:28 |
gyee | but it doesn't seem to do any IceHouse upgrade at all | 21:28 |
jeblair | fungi: the new logstash workers seem to be, uh, stashing logs. so i'm going to delete the old ones | 21:30 |
jeblair | then add the next 8 | 21:30 |
morganfainberg | clarkb, fwiw, i think i'm going to wokr on httpd keystone and ceiliometer during the holidays here | 21:30 |
fungi | jeblair: sounds great | 21:30 |
morganfainberg | clarkb, figured you'd want to know. | 21:30 |
morganfainberg | clarkb, in devstack that is | 21:30 |
fungi | gyee: i've only ever run grenade from the openstack-infra/devstack-gate scripts, so i can't provide much of a suggestion beyond do what d-g does to run grenade' | 21:31 |
gyee | fungi, good to know! let me see if I can duplicate that in my local VM | 21:33 |
openstackgerrit | James E. Blair proposed a change to openstack-infra/config: Remove old logstash worker nodes https://review.openstack.org/62992 | 21:34 |
jeblair | clarkb, mordred, fungi: the launch of worker09 completed without error. thanks. | 21:36 |
fungi | yay! | 21:36 |
*** hashar has quit IRC | 21:36 | |
jeblair | and fast too | 21:36 |
fungi | looks like we're starting to trend snmp oids from the first half of the new workers | 21:40 |
fungi | the slivers i see look mostly nominal | 21:40 |
openstackgerrit | James E. Blair proposed a change to openstack-infra/devstack-gate: Remove strange duplication of setup_host and output https://review.openstack.org/62983 | 21:41 |
*** jhesketh_ has joined #openstack-infra | 21:42 | |
*** tjones has joined #openstack-infra | 21:45 | |
*** tjones has left #openstack-infra | 21:45 | |
*** eharney has quit IRC | 21:46 | |
jeblair | fungi: i just set the default cacti tree to alpha sorting | 21:47 |
*** eharney has joined #openstack-infra | 21:48 | |
fungi | yet another thing i kept meaning to look into but never found the time | 21:48 |
fungi | it was tolerable at 10 servers. i've had to resort to keyword searching for months | 21:48 |
*** senk has joined #openstack-infra | 21:49 | |
reed | fungi, now that the git repo from tipit is in our repo, too, what's next step? | 21:50 |
*** senk1 has joined #openstack-infra | 21:51 | |
jeblair | reed: first, don't commit any more changes to the tipit repo and start using gerrit | 21:51 |
reed | jeblair, makes sense... | 21:52 |
reed | so they need to pull from git.openstack.org, right? | 21:52 |
*** senk1 has quit IRC | 21:53 | |
jeblair | reed, fungi: i imagine two things, possibly in parallel; if the code in the repo is installable, add that to the puppet module for the server | 21:53 |
jeblair | reed, fungi: and spin up the server | 21:53 |
jog0 | jeblair: https://review.openstack.org/#/c/62784/ | 21:53 |
*** senk has quit IRC | 21:53 | |
*** senk has joined #openstack-infra | 21:54 | |
fungi | jeblair: well, the server's already there and deploying that project, but from github at the moment | 21:54 |
fungi | reed: yes, we need to update the puppet configuration vcsrepo object for that project to switch to using a git.o.o origin | 21:54 |
reed | fungi, jeblair, ok, I'm shooting an email to all of you now | 21:54 |
jeblair | reed: oh, then what i've said is wrong. :) | 21:54 |
*** dcramer_ has quit IRC | 21:56 | |
fungi | reed: and from what smarcet was saying earlier, it sounds like he needs to let us know where his application would normally consume database connection details/credentials from (probably a config file?) so that we can template that and inject the appropriate bits into it from hiera | 21:56 |
jeblair | jog0: yeah, i want to kill the whole process group on timeout; not sure how to do that yet | 21:56 |
reed | fungi, clear | 21:56 |
openstackgerrit | Joe Gordon proposed a change to openstack-infra/devstack-gate: Preserve testr temp files https://review.openstack.org/62786 | 21:57 |
fungi | reed: basically the database for openstackid-dev.o.o already exists in a rackspace trove instance, but his app needs to be given the info for how to reach it | 21:57 |
fungi | reed: another to do item is for us to add some additional puppetry to set up https on that machine (i can put a self-signed cert and key for that into hiera as well) | 21:58 |
reed | fungi, great, so you need to know which file to edit? or you need to tell smarcet how to configure his app | 21:58 |
jog0 | jeblair: cool, just wanted to make sure it didn't get forgotton about | 21:59 |
fungi | reed: mainly just want to know where the configuration would go and in what format (an example config file and path for that would suffice) | 21:59 |
*** elasticio has quit IRC | 22:00 | |
reed | fungi, for development not to happen on github anymore, any documentation for them to read? | 22:00 |
fungi | reed: well, it'll be the same thing which happened for the openstack-infra/groups project. that's one tipit's working on as well right (just different devs there)? | 22:01 |
*** MarkAtwood has quit IRC | 22:02 | |
fungi | the development process will be somewhat familiar to smarcet already at this point since he used the same workflow to propose patches to openstack-infra/config, but now he also gets the ability to +2 and approve changes for openstack-infra/openstackid | 22:02 |
clarkb | morganfainberg: cool (though working over the holidays is not required :) ) | 22:03 |
reed | fungi, no, groups is done by marton kiss, he's not affiliated with tipit | 22:03 |
clarkb | jeblair: \o/ | 22:03 |
reed | fungi, does he need to change his remote git repo? | 22:03 |
jog0 | sdague: reviewing your e-r patches now | 22:04 |
clarkb | jeblair: on 09 can we enable crm and see how it handles the extra load? | 22:04 |
jeblair | clarkb: sure | 22:04 |
*** fbo is now known as fbo_away | 22:05 | |
*** vickleford has joined #openstack-infra | 22:05 | |
fungi | reed: yeah, he'll want to change his git remotes from github.com to git.openstack.org | 22:06 |
fungi | reed: and more importantly, to openstack-infra/openstackid | 22:07 |
fungi | as the remote repository name | 22:07 |
*** jcooley_ has joined #openstack-infra | 22:07 | |
*** agordeev has quit IRC | 22:07 | |
fungi | reed: something along the lines of sed -i 's,github.com/blah/idproject,git.openstack.org/openstack-infra/openstackid' .git/config | 22:08 |
fungi | er, missed a trailing , there but you get the idea | 22:08 |
*** dcramer_ has joined #openstack-infra | 22:09 | |
*** rnirmal has quit IRC | 22:09 | |
fungi | reed: i didn't realize marton was working independently on that. very cool | 22:10 |
openstackgerrit | A change was merged to openstack-infra/config: Remove old logstash worker nodes https://review.openstack.org/62992 | 22:10 |
openstackgerrit | A change was merged to openstack-infra/elastic-recheck: allow facets to work at different resolutions https://review.openstack.org/62652 | 22:11 |
*** MarkAtwood has joined #openstack-infra | 22:13 | |
*** lcheng has quit IRC | 22:16 | |
*** mfer has quit IRC | 22:19 | |
*** dkranz has quit IRC | 22:19 | |
*** elasticio has joined #openstack-infra | 22:20 | |
*** dizquierdo has joined #openstack-infra | 22:21 | |
*** kgriffs is now known as kgriffs_afk | 22:23 | |
*** mancdaz is now known as mancdaz_away | 22:23 | |
zaro | clarkb: question about copy console log. | 22:25 |
clarkb | zaro: go for it | 22:25 |
zaro | clarkb: what is the significance of the wait here? https://github.com/jenkinsci/scp-plugin/blob/master/src/main/java/be/certipost/hudson/plugin/SCPRepositoryPublisher.java#L378 | 22:25 |
zaro | clarkb: i mean why 500? because that's the problem. | 22:25 |
clarkb | zaro: it was just a non zero value that was picked | 22:26 |
clarkb | why is 500 a problem? | 22:26 |
zaro | clarkb: seems like that's too long now. | 22:26 |
clarkb | zaro: too long? if it was too long wouldn't the entire log file be copied? | 22:26 |
clarkb | zaro: basically what is happening there is we are yielding to other threads so that the job can finish and have a complete console log | 22:27 |
clarkb | otherwise we don't copy the entire log file (which is what is happening here so the waiting probably isn't working properly) because the job and IO are not complete yet | 22:28 |
zaro | clarkb: i thought getLogText only gets pieces that's why you need to check with isComplete? | 22:29 |
jd__ | mordred: did you just disappear on me after that Python 3 question? | 22:30 |
clarkb | zaro: it only gets pieces because the log isn't complete usually | 22:30 |
*** thomasem has quit IRC | 22:30 | |
zaro | clarkb: actually i haven't figured out the root cause, but let me try some more things. | 22:31 |
zaro | clarkb: just letting you know that dialing that down will get the entire log. | 22:32 |
clarkb | zaro: what are you dialing it down to? | 22:32 |
clarkb | zaro: I wonder if that thread is being killed for some reason when it waits for half a second? | 22:32 |
zaro | clarkb: worked when dialed down to 50. | 22:33 |
clarkb | zaro: how large is your console log? I wonder if that still works when the console log is >6MB | 22:33 |
zaro | clarkb: that's what i'm guessing as well. | 22:33 |
jeblair | clarkb, fungi: worker16 is running crm114 (and has puppet stopped) | 22:34 |
zaro | clarkb: not that large. | 22:34 |
fungi | jeblair: glued to the graph ;) | 22:34 |
*** dstanek has quit IRC | 22:34 | |
zaro | clarkb: only about 50K | 22:35 |
fungi | clarkb: zaro: that was the suggestion jeblair floated... the job ends and the console log collector thread is terminated before it's done retrieving all the data | 22:35 |
jeblair | we've just jumped from 8 to 32 workers, and only 2 of them are running crm114, so i expect a pretty mininmal load (in fact it's mostly idle right now; it can barely pick up a job due to all the competition) | 22:35 |
clarkb | quick run more jobs >_> | 22:36 |
clarkb | fungi: zaro: jeblair: that seems likely. Mean of jenkins to do that, but likely | 22:36 |
jeblair | clarkb, fungi: but we can start changing the config on the other servers and ramp it up. | 22:36 |
fungi | i suppose i could stick a broken python package on the pypi mirror and watch all the approved patches start piling up in the gate ;) | 22:36 |
*** iccha-dinner is now known as iccha | 22:36 | |
* fungi would not *actually* do that of course | 22:37 | |
jeblair | fungi: i think we should send a mail to the -dev mailing list asking people to approve patches without testing and keep bashing recheck until changes land. ;) | 22:37 |
fungi | bwahaha | 22:37 |
clarkb | I wonder if that is because the thread object lives in perform()'s scopr | 22:37 |
clarkb | zaro: ^ | 22:37 |
*** yamahata has quit IRC | 22:39 | |
openstackgerrit | James E. Blair proposed a change to openstack-infra/devstack-gate: Remove strange duplication of setup_host and output https://review.openstack.org/62983 | 22:41 |
sdague | jog0: can you look again there on responses - https://review.openstack.org/#/c/62694/ | 22:42 |
sdague | I'll fix the docs, but I want to get a feel on the functionality changes to work on | 22:42 |
jeblair | jog0: I'm stumped; i've produced the whole stack of shell scripts and tox (including pretty_tox.sh) in a test environment, and timeout does the right thing with process groups | 22:42 |
jeblair | jog0: my guess is that something about running devstack itself changes that | 22:43 |
jeblair | fungi: do you have any ideas? http://logs.openstack.org/84/62784/3/check/check-dg-tempest-dsvm-full/abe3d68/console.html | 22:44 |
jeblair | fungi: note at the bottom after /usr/bin/timeout times out, tempest keeps running | 22:44 |
jeblair | compare to http://logs.openstack.org/81/62781/1/check/check-dg-tempest-dsvm-full/4227047/console.html where putting the timeout on tox kills it much more cleanly | 22:45 |
clarkb | wow you even gave it -9 | 22:45 |
*** senk has quit IRC | 22:46 | |
jeblair | jog0: at any rate, we could just go ahand and live with a few extra tests running after the timeout. i mean, there aren't likely to be many -- after all the job is timing out. | 22:46 |
jog0 | sdague: responded | 22:46 |
zaro | clarkb: so you think the thread for perform is terminated taking down the thread to copy console? | 22:46 |
sdague | jog0: only on the top one :) | 22:47 |
clarkb | zaro: maybe, but a quick read on java threads seems to indicate losing the reference to a thread object doesn't make it go away necessarily. Trying to sort out if that makes the thread GC able | 22:47 |
fungi | jeblair: yeah, i see that... um... maybe the file descriptor is attached but the child is disassociated from the controlling process and in a different process group? | 22:48 |
*** sarob has quit IRC | 22:48 | |
clarkb | zaro: ya doesn't look like java Threads are GCable | 22:49 |
jeblair | fungi: yeah, i made a test env with a wrap script calling a gate script calling sudo calling tox calling pretty_tox.sh calling testr, and everything worked.. | 22:49 |
jog0 | jeblair: I think hte current solution although imperfect is a great starting point and worth going forward with | 22:49 |
jog0 | sdague: just did bottom | 22:49 |
jeblair | fungi: which is why i was looking at devstack suspiciously | 22:50 |
openstackgerrit | Monty Taylor proposed a change to openstack-infra/storyboard: Add SQLalchemy database model https://review.openstack.org/62239 | 22:50 |
fungi | jeblair: if we can identify the process group to target, kill -9 -- -<pgroup> | 22:50 |
clarkb | if at first you don't succeed, clone the jenkins source code :) | 22:50 |
*** dstanek has joined #openstack-infra | 22:51 | |
*** senk has joined #openstack-infra | 22:52 | |
sdague | jog0: ok, so I fundamentally disagree about the job list. I think it's important to keep the unit test and docs fails in the total fail count, because they are often good indicators of infra fails. | 22:53 |
sdague | And while we don't bot them, the queries can match on them | 22:53 |
*** rongze has joined #openstack-infra | 22:53 | |
sdague | so we can bundle them after the fact, like we did with Bug 1260654 | 22:54 |
uvirtbot | Launchpad bug 1260654 in openstack-ci "Could not initialize class jenkins.model.Jenkins$MasterComputer" [Critical,Fix released] https://launchpad.net/bugs/1260654 | 22:54 |
*** gothicmindfood has joined #openstack-infra | 22:55 | |
jog0 | sdague: fair enough, but what about in a seperate category or something | 22:55 |
jog0 | sdague: true | 22:56 |
sdague | if it is causing a gate reset, we really shouldn't ignore it | 22:56 |
openstackgerrit | James E. Blair proposed a change to openstack-infra/devstack-gate: WIP: test timout #2 https://review.openstack.org/62784 | 22:57 |
jeblair | fungi: ^ that's an attempt to see what's going on. and yeah, maybe we can have the top of the gate script write out its pgrp and then have the wrap kill it after the timeout | 22:57 |
*** dims_ has quit IRC | 22:57 | |
jog0 | I agree don't ignore, but I think some sort of differentiation might be worth it | 22:58 |
jog0 | but we can revisit that point later | 22:58 |
jog0 | not worth holding up for that | 22:58 |
*** rongze has quit IRC | 22:59 | |
*** jhesketh_ has quit IRC | 22:59 | |
openstackgerrit | Sean Dague proposed a change to openstack-infra/elastic-recheck: add per job percentages for responsibility of fails https://review.openstack.org/62738 | 23:01 |
openstackgerrit | Sean Dague proposed a change to openstack-infra/elastic-recheck: add support for classification rate https://review.openstack.org/62694 | 23:01 |
*** _david_ has quit IRC | 23:01 | |
sdague | ok that has the doc fixes and the collection iterator, and now I have to figure out how I broke the histogram for the graphs | 23:01 |
clarkb | zaro: AbstractBuild has a cleanUp() method that may be causing the problem | 23:01 |
clarkb | zaro: though that appears to release the workspace lease, clear the env variables, and trigger downstream builds. Nothing that should be killing that thread | 23:04 |
*** vickleford has quit IRC | 23:06 | |
*** SushilKM has quit IRC | 23:09 | |
clarkb | zaro: a quick scan of git log doesn't show anything promising. We may need to attach a debugger | 23:10 |
*** dims_ has joined #openstack-infra | 23:13 | |
*** gothicmindfood has quit IRC | 23:13 | |
clarkb | jeblair: are we ready to try your swap fix for rax performance nodes yet? | 23:14 |
*** senk has quit IRC | 23:14 | |
jeblair | clarkb: if i can ever find it again; it should get resurrected and rechecked | 23:15 |
jeblair | i really hate auto-abandon | 23:15 |
*** MarkAtwood has quit IRC | 23:17 | |
*** esker has quit IRC | 23:17 | |
*** sarob has joined #openstack-infra | 23:18 | |
mordred | jeblair: yeah. I'm liking it less myself as well | 23:18 |
*** mriedem has quit IRC | 23:19 | |
*** senk1 has joined #openstack-infra | 23:19 | |
*** gyee has quit IRC | 23:20 | |
openstackgerrit | Sean Dague proposed a change to openstack-infra/elastic-recheck: correct issue of dangling µs in buckets https://review.openstack.org/63015 | 23:22 |
sdague | jog0: so that one should be looked at quickly | 23:23 |
sdague | it will fix the graphs when it lands | 23:23 |
*** sarob has quit IRC | 23:23 | |
*** dstanek has quit IRC | 23:25 | |
sdague | fungi: thanks | 23:27 |
fungi | sdague: non-ascii in commit subject... uncleeeeean! (seriously though, you're going to wear out your altgr key with all that mooing) | 23:28 |
openstackgerrit | Clark Boylan proposed a change to openstack-infra/devstack-gate: Use the ephemeral disk on rackspace performance nodes https://review.openstack.org/60347 | 23:28 |
clarkb | jeblair: ^ rebased for you | 23:28 |
*** mrodden has quit IRC | 23:28 | |
sdague | fungi: we're in a utf8 world man :) | 23:30 |
*** weshay has quit IRC | 23:30 | |
sdague | I also thought it would be a nice test of our pipeline | 23:30 |
clarkb | fungi: sdague: my weecaht in screen can render it properly now so I don't care :) | 23:30 |
sdague | yep | 23:30 |
sdague | it's been fine in everything I've looked at it in | 23:31 |
fungi | sdague: clarkb: looks right to me in gerrit and gerritbot/irc | 23:31 |
fungi | we win at international characterset encoding! | 23:31 |
sdague | you'll appreciate the utf8 char I used for utf8 testing image names | 23:31 |
clarkb | sdague: screen requires special bit flipping or options to do unicode | 23:31 |
*** MarkAtwood has joined #openstack-infra | 23:31 | |
fungi | sdague: snowman? get with the times... there's a beer mug glyph with an official codepoint | 23:32 |
fungi | jeblair: timeout: failed to run command `/opt/stack/devstack-vm-gate.sh': No such file or directory | 23:32 |
sdague | https://review.openstack.org/#/c/61747/9/tempest/api/compute/images/test_images_oneserver.py | 23:32 |
sdague | Masayuki's comment is funny too | 23:32 |
*** UtahDave has quit IRC | 23:32 | |
sdague | "I'm not sure this unicode character is "valid" for the "Enterprise" though :)." | 23:33 |
fungi | hahaha | 23:33 |
fungi | i think i have actually seen that one before as well | 23:33 |
*** dcramer_ has quit IRC | 23:39 | |
marun | I know I've asked before, but how do I go about adding a new gate job (functional) for neutron? It will need to run with sudo privileges (iirc running as the tempest user should work). It will also need things like openvswitch installed (hopefully the devstack vm prep will do that?). | 23:43 |
marun | sdague: ^ | 23:43 |
openstackgerrit | A change was merged to openstack-infra/elastic-recheck: correct issue of dangling µs in buckets https://review.openstack.org/63015 | 23:44 |
clarkb | marun: give me a minute I will link you to some commit shas that have done it | 23:44 |
marun | clarkb: danke | 23:45 |
clarkb | hrm all of them are old enough that maybe better to just describe the current process | 23:46 |
*** rcleere has quit IRC | 23:46 | |
*** romcheg has quit IRC | 23:46 | |
anteaya | marun: this adds an existing job, devstack exercises: https://review.openstack.org/#/c/62930/ | 23:46 |
clarkb | eg not sufficient to give you one diff, need many | 23:46 |
clarkb | oh anteaya has one perfect | 23:47 |
clarkb | marun: see that change | 23:47 |
anteaya | it doesn't write the job though | 23:47 |
anteaya | just uses an existing one | 23:47 |
clarkb | right but the glue is there | 23:47 |
clarkb | marun: if you look at the other jobs in those files some of them set export hook functions which run at different times in the devstack-gate process | 23:48 |
clarkb | currently you can do post devstack pre test, hook to run tests, and hook after tests iirc | 23:48 |
*** jergerber has quit IRC | 23:49 | |
marun | clarkb: hmm. this test won't require services to target, just that some of the neutron os dependencies like ovs have been installed | 23:50 |
clarkb | marun: devstack vm prep is basically run devstack | 23:51 |
clarkb | marun: your other option is to have a thing do the prep itself | 23:51 |
marun | clarkb: I was under the impression that installing prereqs and then snapshotting the resulting vm was a part of the jenkins pipeline. | 23:52 |
marun | clarkb: or do I just have an active imagination? | 23:52 |
fungi | heading out to dinner, back on later | 23:53 |
clarkb | marun: close. we cache the prereqs so that devsatck can run as quickly as possible. but installation is done by devstack | 23:54 |
marun | clarkb: ah, gotcha | 23:54 |
marun | clarkb: so the installation has to happen as part of the job. I guess I'll have to see if I can install things without doing a full devstack run. | 23:55 |
clarkb | correct, installation would need to be part of the job | 23:55 |
clarkb | marun: fwiw the pbr integration test which all it does is install openstack stuff with pbr in a variety of ways uses devstack to make sure all of the dependencies are in place | 23:56 |
*** dcramer_ has joined #openstack-infra | 23:56 | |
marun | clarkb: ok, cool. | 23:56 |
sdague | marun: so if you only care about neutron in this job, you should be able to set the service list pretty small in the devstack config yuo'd use for it | 23:58 |
sdague | so you'd use devstack to install, but only keystone + neutron | 23:59 |
*** prad has quit IRC | 23:59 | |
sdague | the devstack install time is about 2 minutes for all of it, so you could probably get to < 60s overhead to do that | 23:59 |
marun | sdague: I don't actually want anything running, though | 23:59 |
openstackgerrit | Russell Bryant proposed a change to openstack-infra/config: zuul status: allow changes to be collapsed https://review.openstack.org/63018 | 23:59 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!