| jeblair | i mean, how does a change to a library actually get tested in devstack? | 00:00 |
|---|---|---|
| clarkb | jeblair ya you need a job to do it from git for glance_store | 00:00 |
| jeblair | that doesn't make any sense... | 00:00 |
| *** ashleighfarnham has quit IRC | 00:00 | |
| jhesketh | jeblair: looking | 00:00 |
| clarkb | jeblair tgere should be prior art for the clients | 00:01 |
| clarkb | iirc you set install from git to ZUUL_PROJECT | 00:01 |
| *** puranamr has quit IRC | 00:01 | |
| jeblair | clarkb: i don't see the devstack jobs running on client changes | 00:02 |
| jeblair | i have this suspicion that basically any libraries aren't doing any worthwhile integration testing which is why we get gate wedges all the time | 00:03 |
| *** annegentle_ has joined #openstack-infra | 00:03 | |
| madhu_ak | hello, I would like to have a core reviewer from infra to approve this one: https://review.openstack.org/#/c/199754/ | 00:04 |
| jeblair | there really should be an "install everything from git" version of devstack, since that's sort of the point | 00:04 |
| *** EricGonczer_ has joined #openstack-infra | 00:04 | |
| *** patrickeast has quit IRC | 00:05 | |
| *** Sukhdev has joined #openstack-infra | 00:05 | |
| madhu_ak | anteaya: could you have a look into it ^^ thanks | 00:05 |
| *** puranamr has joined #openstack-infra | 00:06 | |
| openstackgerrit | James E. Blair proposed openstack-infra/project-config: Add ceph devstack job to glance_store https://review.openstack.org/203299 | 00:06 |
| *** ddieterl_ has quit IRC | 00:06 | |
| jeblair | mriedem, clarkb: ^ sadly, i think that will do nothing. | 00:06 |
| mriedem | jeblair: doesn't it have to be one of the 'src' jobs? | 00:06 |
| *** larainema has quit IRC | 00:06 | |
| clarkb | ya those jobs | 00:06 |
| *** ayoung has joined #openstack-infra | 00:07 | |
| *** tsekiyam_ has joined #openstack-infra | 00:07 | |
| *** larainema has joined #openstack-infra | 00:07 | |
| *** puranamr has quit IRC | 00:07 | |
| *** puranamr has joined #openstack-infra | 00:07 | |
| jeblair | oh 'gate-tempest-dsvm-neutron-src-{name}' is run on clients | 00:08 |
| *** annegentle_ has quit IRC | 00:08 | |
| jeblair | so we could do a src-glance_store and that would work.... | 00:09 |
| jeblair | but this still sort of rubs me the wrong way, because we're testing that change in isolation from other clients... | 00:09 |
| jeblair | i'm starting to think the right apporach is to run two devstack jobs in the integrated gate -- install all libs from pypi, and install all libs from git | 00:09 |
| *** mtanino has quit IRC | 00:10 | |
| *** tsekiyama has quit IRC | 00:10 | |
| lifeless | jeblair: so yes, this is the 'and we should run the jobs that can break on new packages on proposed changes to g-r' | 00:11 |
| *** tsekiyam_ has quit IRC | 00:11 | |
| lifeless | jeblair: its evidence the design works (we weren't broken until upper-constraints changed' | 00:11 |
| lifeless | jeblair: but that thet set of tests we're gating on for that is too slim | 00:12 |
| jeblair | lifeless: oh, so you think a ceph job on reqs would have helped? | 00:12 |
| *** rm_work is now known as rm_work|away | 00:12 | |
| lifeless | jeblair: it would have voted 'broken' on the upper-requirements.txt change | 00:12 |
| jeblair | mriedem, clarkb: 'lib-forward-testing' is already on glance_store | 00:13 |
| lifeless | jeblair: so yes, I do. It wouldn't have stopped a broken glance-store being *released*, but it would have made it not a fire-drill | 00:13 |
| clarkb | jeblair but that doesnt do ceph | 00:13 |
| jeblair | mriedem, clarkb: oh, but of course, it's not ceph | 00:13 |
| jeblair | right | 00:13 |
| jeblair | lifeless: yep. so maybe add jobs to both to prevent both :) | 00:13 |
| lifeless | jeblair: sounds good to me | 00:14 |
| lifeless | jeblair: assuming ceph isn't generally flaky | 00:14 |
| jeblair | except i still have no idea how to test this on glance_store | 00:14 |
| lifeless | jeblair: on glance-store changes? | 00:15 |
| clarkb | need a new src ceph job | 00:15 |
| lifeless | jeblair: that would require installing ceph with glance-store from-git | 00:15 |
| clarkb | that does the src client thing and ceph | 00:15 |
| lifeless | jeblair: I haven't looked into the infra-wire-up of that | 00:15 |
| jeblair | yeah, i find that frustrating :) | 00:15 |
| *** dannywilson has quit IRC | 00:15 | |
| lifeless | jeblair: the requirements job should be trivial, just the normal ceph dvsm job | 00:15 |
| *** madhu_ak has quit IRC | 00:17 | |
| openstackgerrit | Merged openstack-infra/elastic-recheck: Add query for ceph glance_store bug 1475811 https://review.openstack.org/203296 | 00:18 |
| openstack | bug 1475811 in Glance "gate-tempest-dsvm-full-ceph fails to upload image with "TypeError: an integer is required"" [Critical,In progress] https://launchpad.net/bugs/1475811 - Assigned to Matt Riedemann (mriedem) | 00:18 |
| *** bharath has quit IRC | 00:18 | |
| clarkb | maybe ew can do something like lib from git always == ZUUL_PROJECT | 00:20 |
| clarkb | then collapse the jobs | 00:20 |
| openstackgerrit | James E. Blair proposed openstack-infra/project-config: Add ceph devstack job to glance_store https://review.openstack.org/203299 | 00:20 |
| jeblair | clarkb, mriedem: ^ using existing src pattern | 00:21 |
| jeblair | clarkb: yeah, that's an idea, though i think what we have is two goals that can't be tested in the same job: make sure the released world doesn't break, and make sure we don't introduce bugs that will break things when released. | 00:22 |
| jeblair | clarkb: which is why i'm thinking of a "libs from pypi" job and an "all libs from git" job | 00:22 |
| lifeless | yes | 00:23 |
| lifeless | I support this thinking | 00:23 |
| clarkb | jeblair: for the first thing we get that whenever we make changes to not the lib (and by the constraints enforcement and bumping) for the second we get that by testing changes to the lib itself from source | 00:23 |
| lifeless | however, I know sdague has spent a bunch of time around what-jobs-when etc, so perhaps loop him in ? | 00:23 |
| jeblair | clarkb: then there's real integration testing again, since the "one lib at a time" job doesn't do integration testing. | 00:23 |
| *** shashank_hegde has quit IRC | 00:23 | |
| *** marzif__ has joined #openstack-infra | 00:23 | |
| jeblair | at least, it doesn't integrate with other changes in-flight | 00:23 |
| jeblair | so we can still wedge | 00:23 |
| jeblair | lifeless: yep | 00:24 |
| clarkb | jeblair: right, the assumpton was we only need to make sure the changing side doesn't break things (but that isn't necessarily true I guess) | 00:24 |
| lifeless | one thing I hesitate around here, for instance, is that a lot of effort was spent to decouple the various things and depend only on releases | 00:24 |
| lifeless | and unwinding that seems to be against that consensus | 00:24 |
| lifeless | whereas checking on the re-entry doesn't | 00:24 |
| lifeless | and with dhellmann's cunning plan to have gates around releases | 00:25 |
| lifeless | we can make sure we don't release broken things | 00:25 |
| lifeless | without making every git commit have to run dvsm from-git | 00:25 |
| jeblair | lifeless: i recognize the value of that and don't want to undo it. i want to keep that and add back in the ability to avoid landing changes that break things. | 00:25 |
| *** bharath has joined #openstack-infra | 00:25 | |
| lifeless | erm every library commit have to run dvsm from-git against $other | 00:25 |
| *** patrickeast has joined #openstack-infra | 00:25 | |
| clarkb | I have cleaned up my held nodes | 00:25 |
| jeblair | lifeless: i'm not familiar with a plan to run gates around releases | 00:26 |
| lifeless | jeblair: dhellmann has a repo now where library releases are described in a yaml file | 00:26 |
| lifeless | jeblair: he plans to make a job that does stuff with the proposed sha before pushing the tag | 00:26 |
| lifeless | jeblair: so that the tag-pushing becomes driven from approvals to that repo | 00:26 |
| jeblair | lifeless: yep: http://specs.openstack.org/openstack-infra/infra-specs/specs/centralize-release-tagging.html | 00:26 |
| sigmavirus24 | lifeless: wouldn't hurt to know about ceph problems before release time though | 00:27 |
| lifeless | sigmavirus24: sure | 00:27 |
| lifeless | and hey, I'm all for more testing | 00:27 |
| jeblair | lifeless: i provided a lot of feedback on that spec, but don't recall anything in it about testing things before release | 00:27 |
| *** marzif_ has quit IRC | 00:27 | |
| lifeless | My hesitation here is that I don't want to accidentally undo stuff other folk spent a bunch of time putting together | 00:27 |
| lifeless | thats all | 00:27 |
| jeblair | lifeless: yep, i think my suggestion is additive | 00:27 |
| *** ddieterly has joined #openstack-infra | 00:28 | |
| jhesketh | clarkb: Given people are still busy, I suggest we wait an hour or so before merging those os-loganalyze updates if that works for you? (I'm also happy to do it later today when things are even quieter as worst case reverting it is very straight forward) | 00:28 |
| lifeless | I don't think I can proxy sdague's thinking on this sufficiently well | 00:28 |
| lifeless | so, I'm not going to support or oppose :) | 00:28 |
| clarkb | jhesketh: works fine for me | 00:28 |
| clarkb | let me go double check the changes now while I am thinking about it | 00:28 |
| clarkb | my +2s are still on most of the changes | 00:29 |
| jhesketh | okay thanks :-) | 00:29 |
| jeblair | lifeless: best to let him do his own thinking. :) | 00:29 |
| lifeless | jeblair: yep | 00:29 |
| lifeless | jeblair: the requirements gate on ceph though, I can support, since it was designed in. sdague was worried about running *everything* due to the additive nature of flaky tests. | 00:30 |
| *** bharath has quit IRC | 00:30 | |
| lifeless | jeblair: but not about *some increase* - and its a knob we can dial up or down too. | 00:30 |
| jeblair | lifeless: the main argument for not installing libs from git is that we kept landing changes to things that used libs that required those unreleased changes, so we had to make sure servers could run with the released versions of dependencies they expressed. | 00:31 |
| jeblair | this is a good thing for us to test and i do not want to stop doing it | 00:31 |
| openstackgerrit | James E. Blair proposed openstack-infra/project-config: Add ceph devstack job to requirements https://review.openstack.org/203303 | 00:34 |
| jeblair | actually, let me reparent that | 00:34 |
| clarkb | jhesketh: reading the diff between the old change and the new change looks like the only real updates are where you deal with the trailing slashes as thats newer than the reverted code | 00:34 |
| clarkb | jhesketh: that sound right to you? | 00:34 |
| openstackgerrit | James E. Blair proposed openstack-infra/project-config: Add ceph devstack job to requirements https://review.openstack.org/203303 | 00:34 |
| jeblair | lifeless, mriedem: ^ | 00:34 |
| *** s0nea has quit IRC | 00:34 | |
| jhesketh | clarkb: yep, that was the merge conflict | 00:34 |
| *** EricGonczer_ has quit IRC | 00:36 | |
| *** ajmiller has quit IRC | 00:36 | |
| openstackgerrit | James E. Blair proposed openstack-infra/project-config: Add ceph devstack job to glance_store https://review.openstack.org/203299 | 00:37 |
| lifeless | jeblair: absolutely, great to hear that: knowing we work with latest-released is very important to me ;) | 00:37 |
| mriedem | jeblair: i think we should just do https://review.openstack.org/#/c/203299/ | 00:37 |
| mriedem | jeblair: otherwise other optional backends will be clamoring for gating on requirements too like glusterfs and sheepdog | 00:37 |
| mriedem | that came up when we pushed for gating ceph on cinder | 00:37 |
| *** s0nea has joined #openstack-infra | 00:38 | |
| openstackgerrit | James E. Blair proposed openstack-infra/project-config: Add ceph devstack job to requirements https://review.openstack.org/203303 | 00:38 |
| jeblair | mriedem: yeah, that's a good point. i won't be offended if we decide not to do that. maybe collect a few more thoughts on that next week? | 00:39 |
| jeblair | mriedem: you should re-minus-one that | 00:39 |
| jeblair | oh good you just did :) | 00:40 |
| mriedem | woot | 00:40 |
| clarkb | part of me doesn't think that should be a problem | 00:40 |
| mriedem | i worry about cat herding | 00:40 |
| clarkb | I mean we have chosen the mysql + qemu + nova net is sort of the blessed thing | 00:40 |
| clarkb | no one is screaming that postgres should be tested more | 00:40 |
| clarkb | or that we should use lxc | 00:40 |
| jeblair | some neighborhood cats are contesting whose territory my yard is, and i have been doing a lot of literal cat herding lately. | 00:41 |
| *** _ody has joined #openstack-infra | 00:41 | |
| clarkb | we can make good decisions for effective testing independent of cat herding I hope | 00:41 |
| clarkb | jeblair: you need to assert yourself as alpha cat | 00:41 |
| * jeblair dons alpha cat costume | 00:41 | |
| *** marzif__ has quit IRC | 00:41 | |
| *** rfolco has quit IRC | 00:42 | |
| jeblair | clarkb: do you have outstanding q's on the zuul change? | 00:42 |
| clarkb | jeblair: just if two jobs had failed will the second be reset to skipped in the live reconfiguration after fail case | 00:42 |
| clarkb | jeblair: when it should preserve the second jobs fail instead | 00:42 |
| jeblair | clarkb: that should only affect job trees, which won't run a child job if the parent fails, so the only possibilities are: SUCCESS->(any) or FAILED->SKIPPED | 00:43 |
| clarkb | jeblair: oh right | 00:44 |
| clarkb | jeblair: this is when you have pipeline:\n - foo:\n - bar | 00:44 |
| clarkb | if foo fails bar doesn't run | 00:44 |
| jeblair | clarkb: so on re-enqueue, if the parent succeeded, it will basically noop the parent, then the same thing will happen for children. if the parent failed, it will mark all the children (which have not run) as skipped | 00:44 |
| jeblair | clarkb: yep | 00:44 |
| jeblair | clarkb: in the tests, the jobs "test-*" are children of "test-merge" | 00:45 |
| jeblair | (we used to run that way!) | 00:45 |
| clarkb | ya | 00:45 |
| clarkb | looks like tests failed again and jhesketh has comments | 00:45 |
| *** baoli has quit IRC | 00:46 | |
| *** _ody has quit IRC | 00:46 | |
| clarkb | http://www.oregonlive.com/health/index.ssf/2015/07/salmonella_outbreak_in_portlan.html and you thought your conference illness was bad | 00:46 |
| jeblair | ugh | 00:47 |
| clarkb | jhesketh: I am +2 on the os-loganalyze stack | 00:48 |
| jhesketh | cheers | 00:48 |
| *** baoli has joined #openstack-infra | 00:48 | |
| clarkb | huh it thinks change A which should merge hasn't merged | 00:49 |
| jeblair | http://logs.openstack.org/81/203281/2/check/gate-zuul-python27/a3de3a4/console.html#_2015-07-17_23_40_36_532 | 00:49 |
| jeblair | that's weird. | 00:49 |
| clarkb | jeblair: do we have to release the gerrit events? is it possile that B is processed first due to some reace so A ends up sticking around and not B? | 00:50 |
| jeblair | no i think http://logs.openstack.org/81/203281/2/check/gate-zuul-python27/a3de3a4/console.html#_2015-07-17_23_40_36_532 is the cause | 00:50 |
| clarkb | oh weird indeed | 00:51 |
| *** puranamr has quit IRC | 00:54 | |
| *** puranamr has joined #openstack-infra | 00:55 | |
| *** xyang1 has quit IRC | 00:55 | |
| *** sigmavirus24 is now known as sigmavirus24_awa | 00:58 | |
| *** pvaneck has quit IRC | 00:59 | |
| *** esmute has quit IRC | 00:59 | |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul: Handle adding a job to a failing change during reconfig https://review.openstack.org/203281 | 01:01 |
| jeblair | addressed jhesketh's comments; did nothing about the weird git thing. | 01:01 |
| *** puranamr_ has joined #openstack-infra | 01:01 | |
| *** puranamr_ has quit IRC | 01:02 | |
| *** puranamr has quit IRC | 01:05 | |
| jeblair | yeah, so that's a really weird flake. | 01:05 |
| clarkb | lgtm | 01:05 |
| Sukhdev | clarkb jeblair: can you please look at this paste especially the last command - http://paste.openstack.org/show/385075/ | 01:08 |
| Sukhdev | clarkb jeblair: what am I doing wrong? | 01:08 |
| clarkb | you don't have git review installed | 01:10 |
| Sukhdev | clarkb: oh - I must have accidentally removed it while trying to clean other packages - thanks for pointing it out | 01:11 |
| * jeblair goes in search of avocados | 01:11 | |
| *** SumitNaiksatam has quit IRC | 01:12 | |
| openstackgerrit | Jeremy Stanley proposed openstack-infra/bindep: Add profile to the usage text https://review.openstack.org/203272 | 01:13 |
| clarkb | fungi: danke | 01:13 |
| *** geoffarnold has joined #openstack-infra | 01:13 | |
| *** baoli has quit IRC | 01:14 | |
| *** baoli has joined #openstack-infra | 01:15 | |
| openstackgerrit | Sukhdev Kapur proposed openstack-infra/project-config: Updating ACLs for networking-arista https://review.openstack.org/203305 | 01:16 |
| *** mriedem1 has joined #openstack-infra | 01:17 | |
| *** unicell has quit IRC | 01:18 | |
| openstackgerrit | Merged openstack-infra/zuul: Handle adding a job to a failing change during reconfig https://review.openstack.org/203281 | 01:18 |
| *** mriedem has quit IRC | 01:19 | |
| fungi | lifeless: mriedem: i agree if the gate-tempest-dsvm-full-ceph job is gating on tempest and/or devstack then requirements makes sense to add too | 01:20 |
| fungi | (or i guess nova+cinder+glance... at least) | 01:20 |
| mtreinish | fungi: I don't think it's voting on tempest, and fwiw I don't really think it makes sense there | 01:21 |
| fungi | in this case the upper-constraints.txt change from ==0.6.0 to 0.7.0 would have been tested by the ceph job (assuming it's running with constraints enabled) and failed, preventing it from landing | 01:23 |
| * fungi is now noticing people have already said the things he's saying and stops ;) | 01:25 | |
| *** Sukhdev has quit IRC | 01:28 | |
| openstackgerrit | Matthew Treinish proposed openstack-infra/subunit2sql: Add migration to add indexes on common search patterns https://review.openstack.org/196194 | 01:31 |
| openstackgerrit | Matthew Treinish proposed openstack-infra/subunit2sql: Don't try to create indexes twice https://review.openstack.org/196193 | 01:31 |
| openstackgerrit | Matthew Treinish proposed openstack-infra/subunit2sql: Compact existing schema and start 1.x.x release branch https://review.openstack.org/203252 | 01:31 |
| openstackgerrit | Matthew Treinish proposed openstack-infra/subunit2sql: Add db migration to align column size to 255 and drop FKs https://review.openstack.org/203251 | 01:31 |
| *** woodster_ has quit IRC | 01:32 | |
| openstackgerrit | Matthew Treinish proposed openstack-infra/subunit2sql: Compact existing schema and start 1.x.x release branch https://review.openstack.org/203252 | 01:37 |
| openstackgerrit | Matthew Treinish proposed openstack-infra/subunit2sql: Add db migration to align column size to 255 and drop FKs https://review.openstack.org/203251 | 01:37 |
| *** jasondotstar has joined #openstack-infra | 01:38 | |
| *** jasondotstar has quit IRC | 01:38 | |
| *** jasondotstar has joined #openstack-infra | 01:38 | |
| *** geoffarnold has quit IRC | 01:40 | |
| *** jasondotstar has quit IRC | 01:40 | |
| *** earlephilhower has quit IRC | 01:40 | |
| lifeless | fungi: tempest and devstack are not library changes though | 01:43 |
| lifeless | fungi: and ceph is arguably our most popular block volume layer IIRC | 01:43 |
| lifeless | fungi: so I'm not sure whether tempest and devstack make sense, but I certainly think requirements does entirely independently | 01:43 |
| lifeless | has https://review.openstack.org/#/c/199782/ gone into lala land? | 01:45 |
| fungi | it's gone into lala-two-neutron-changes-ahead-of-it-in-the-gate-land | 01:46 |
| fungi | jenkins says eta 17 minutes to merge, assuming neither of those choke ahead of it | 01:46 |
| fungi | and yeah, if nova, cinder and glance have the ceph job voting on them then chances are requirements is also a sensible add | 01:48 |
| fungi | and 199782 is now up front | 01:49 |
| fungi | just needs to pass the neutron-full job and it's home-free | 01:49 |
| fungi | looks like it's succeeded and reporting now | 01:50 |
| openstackgerrit | Merged openstack/requirements: Move integration test dependencies into bindep https://review.openstack.org/199782 | 01:51 |
| fungi | lifeless: there ^ we go! | 01:51 |
| fungi | ahead of schedule even | 01:51 |
| * fungi should be a sports commentator | 01:51 | |
| fungi | or maybe get some sleep instead | 01:52 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/diskimage-builder: Updated from global requirements https://review.openstack.org/201852 | 01:52 |
| *** annegentle_ has joined #openstack-infra | 01:52 | |
| *** ajmiller has joined #openstack-infra | 01:53 | |
| openstackgerrit | Matthew Treinish proposed openstack-infra/subunit2sql: SUPER WIP: Stop using oslo anything https://review.openstack.org/203308 | 01:56 |
| *** unicell has joined #openstack-infra | 01:57 | |
| *** annegentle_ has quit IRC | 01:57 | |
| openstackgerrit | Matthew Treinish proposed openstack-infra/subunit2sql: SUPER WIP: Stop using oslo anything https://review.openstack.org/203308 | 01:57 |
| mtreinish | jeblair: ^^^ if you have the time or energy to take that over at some point in the future that would be awesome | 02:00 |
| *** unicell1 has joined #openstack-infra | 02:01 | |
| mtreinish | because I have no idea when I'll be able to dedicate serious, not mildly drunk, thought to it | 02:01 |
| *** unicell has quit IRC | 02:01 | |
| fungi | drunkgramming | 02:02 |
| *** sputnik13 has quit IRC | 02:04 | |
| *** ajmiller has quit IRC | 02:06 | |
| *** _ody has joined #openstack-infra | 02:07 | |
| jeblair | mtreinish: thanks! marked for later. | 02:07 |
| jeblair | in the mean time... oops, i put bacon in my guacamole. i better eat it before anyone notices. | 02:08 |
| jhesketh | clarkb: I'll merge the os-loganalyze change(s) now unless you have any objections | 02:08 |
| fungi | i totally had guac tonight | 02:08 |
| fungi | but did not make it myself, so baconless :( | 02:08 |
| jeblair | fungi: better luck next time? | 02:09 |
| fungi | ganbatte kudssai | 02:09 |
| fungi | er, kudasai | 02:09 |
| *** Kennan2 has quit IRC | 02:10 | |
| *** shashank_hegde has joined #openstack-infra | 02:10 | |
| clarkb | jhesketh go for it though I am semi afk for about half an hour while I feed babies back after | 02:10 |
| jhesketh | clarkb: okay, I'll wait a few min and then punch the button (it'll still take some time to apply) | 02:11 |
| clarkb | kk | 02:11 |
| *** FallenPegasus has joined #openstack-infra | 02:11 | |
| *** yamahata has quit IRC | 02:12 | |
| *** baoli has quit IRC | 02:12 | |
| *** FallenPe_ has joined #openstack-infra | 02:13 | |
| *** baoli has joined #openstack-infra | 02:15 | |
| *** baoli has quit IRC | 02:15 | |
| *** FallenPegasus has quit IRC | 02:16 | |
| fungi | took some photos at the house tonight: http://fungi.yuggoth.org/tmp/house/ | 02:20 |
| clarkb | nice | 02:21 |
| clarkb | you face west over the sound/bay/notocean? | 02:22 |
| *** baoli has joined #openstack-infra | 02:23 | |
| *** baoli has quit IRC | 02:23 | |
| *** Kennan has joined #openstack-infra | 02:23 | |
| *** mriedem1 has quit IRC | 02:24 | |
| *** esmute has joined #openstack-infra | 02:27 | |
| fungi | yep, safer there | 02:27 |
| *** EricGonczer_ has joined #openstack-infra | 02:28 | |
| fungi | except for when a storm forces the ocean through the inlet and up into the sound/bay/notocean | 02:28 |
| fungi | where it then tries to escape by carving new inlets | 02:28 |
| openstackgerrit | Merged openstack-infra/os-loganalyze: Reapply tidy up generators into contained objects https://review.openstack.org/185804 | 02:29 |
| fungi | also the ocean-facing beaches here are public property, so you can't have your own dock or boat ramp | 02:30 |
| *** FallenPe_ has quit IRC | 02:31 | |
| *** shashank_hegde has quit IRC | 02:31 | |
| *** esmute has quit IRC | 02:31 | |
| *** SumitNaiksatam has joined #openstack-infra | 02:34 | |
| *** harlowja_at_home has joined #openstack-infra | 02:34 | |
| jhesketh | clarkb, fungi: the new osloganalyze has merged and apache restarted successfully | 02:35 |
| jhesketh | things appear to be good for me | 02:35 |
| fungi | awesome | 02:35 |
| fungi | that's been a long time coming | 02:35 |
| fungi | "" | 02:35 |
| fungi | hrm, wrong terminal, clearly | 02:36 |
| clarkb | neat sp you applied by hand? | 02:36 |
| clarkb | or did puppet timing work out well | 02:36 |
| jhesketh | sp? | 02:36 |
| jhesketh | puppet restarted the service for us | 02:36 |
| clarkb | *so | 02:36 |
| jhesketh | nope, nothing by hand | 02:36 |
| clarkb | ya but puppet is on a ~45 minute cycle | 02:36 |
| jhesketh | clarkb: not on this box http://puppetdb.openstack.org/node/static.openstack.org | 02:37 |
| jhesketh | it's 15min | 02:37 |
| jhesketh | err, no, it's 45 | 02:37 |
| jhesketh | but we were lucky with the timing being next to the merge | 02:37 |
| clarkb | looks like timing just worked | 02:37 |
| clarkb | yup | 02:37 |
| clarkb | do we want to apply the others now too? | 02:38 |
| jhesketh | yep, I think so... shall we do one at a time? | 02:38 |
| clarkb | sure | 02:38 |
| clarkb | and can run `sudo puppet agent --test` to trigger puppet quicker on static.o.o | 02:39 |
| openstackgerrit | Merged openstack-infra/os-loganalyze: Allow the config to set the filter and view https://review.openstack.org/173328 | 02:40 |
| jhesketh | clarkb: okay running | 02:41 |
| *** dims_ has quit IRC | 02:41 | |
| jhesketh | clarkb: okay and that's in and working | 02:42 |
| jhesketh | onto the next one | 02:42 |
| *** achanda has joined #openstack-infra | 02:42 | |
| openstackgerrit | Merged openstack-infra/os-loganalyze: Add support for setting filters and views per file https://review.openstack.org/173329 | 02:44 |
| *** dims_ has joined #openstack-infra | 02:44 | |
| jhesketh | and puppet applied on that one | 02:44 |
| jhesketh | and all working | 02:45 |
| *** asselin has joined #openstack-infra | 02:45 | |
| clarkb | do we have a change to config passthrough for anythibg not matching the existibg rules ib apache? | 02:45 |
| clarkb | dont have to do that now but thats next step right? | 02:45 |
| jhesketh | I don't think we have that change ready | 02:46 |
| openstackgerrit | litong01 proposed openstack-infra/project-config: Add new cloud-monitor repository for OpenStack monitoring service https://review.openstack.org/203259 | 02:47 |
| clarkb | good thing for next week | 02:48 |
| jhesketh | yep | 02:48 |
| jhesketh | right now I want to tidy up the debugging from swift because the apache error.log is 17GB | 02:48 |
| jhesketh | it's an easy fix | 02:48 |
| *** asselin__ has joined #openstack-infra | 02:48 | |
| clarkb | ok | 02:49 |
| jhesketh | clarkb: basically because we try to fetch from swift and then do other things if that fails, this exception is clobbering the error.log http://git.openstack.org/cgit/openstack-infra/os-loganalyze/tree/os_loganalyze/generator.py#n140 | 02:49 |
| jhesketh | but that's still a useful exception so need to filter out if it's a 404 or something else | 02:51 |
| *** asselin has quit IRC | 02:52 | |
| *** baoli has joined #openstack-infra | 02:52 | |
| *** baoli has quit IRC | 02:52 | |
| *** baoli has joined #openstack-infra | 02:52 | |
| *** baoli has quit IRC | 02:53 | |
| clarkb | gotcha | 02:53 |
| *** dims_ has quit IRC | 02:54 | |
| *** SumitNaiksatam has left #openstack-infra | 02:56 | |
| *** baoli has joined #openstack-infra | 02:57 | |
| lifeless | fungi: thanks, for some reason I was blind when reading zuul | 02:59 |
| lifeless | I wish gerrit showed the zuul status live | 02:59 |
| *** jerryz has quit IRC | 03:01 | |
| harlowja_at_home | lifeless, https://github.com/harlowja/gerrit_view/#czuul should still work (last time i checked); if u want something to view specific projects (in a console...) | 03:02 |
| harlowja_at_home | although it would be nice to have zuul live-status on gerrit somehow | 03:02 |
| *** EricGonczer_ has quit IRC | 03:02 | |
| *** boris-42 has quit IRC | 03:02 | |
| fungi | lifeless: we sort of had that for a bit but all the queries killed zuul. we've now got a less shotgun-oriented status api in zuul so that our javascript overlay can make more lightweight requests, though it's not fully in place yet (last i saw there's still a toggle in the javascript disabling it for testing purposes) | 03:02 |
| fungi | sdague was working on it off and on | 03:03 |
| harlowja_at_home | fungi, i thought the zuul webpage was cached (accurate to every 5 seconds or something) anyway (so hopefully most queries just hit the cache?) | 03:05 |
| *** larainema has quit IRC | 03:05 | |
| fungi | harlowja_at_home: yes, however the sheer volume being queried by hundreds of devs every several seconds multiplied by the number of browser tabs they have open to gerrit changes... | 03:06 |
| fungi | it was pretty bad | 03:06 |
| harlowja_at_home | :-/ | 03:06 |
| harlowja_at_home | all served by 1 eventlet greenthread, lol :-P | 03:07 |
| fungi | at the moment, the full status.json is 144k of data | 03:08 |
| openstackgerrit | Joshua Hesketh proposed openstack-infra/os-loganalyze: Surpress 404 errors from swift https://review.openstack.org/203311 | 03:09 |
| jhesketh | clarkb: ^ | 03:09 |
| harlowja_at_home | fungi, ya, every 30 seconds https://github.com/harlowja/gerrit_view/#czuul will try to grab it (and parse, and redisplay) | 03:09 |
| harlowja_at_home | but i'd imagine a webpage viewing 1 review would hopefully be like 1k (if that) | 03:11 |
| fungi | whereas when you're looking at the gerrit webui in the context of a single change, you can query /status/change/203148,2 for ~4k of data | 03:12 |
| harlowja_at_home | nice | 03:13 |
| clarkb | jhesketh +2 maybe fungi will approve? | 03:13 |
| fungi | not 1k, but still a lot less than 144k (and we're not even busy) | 03:13 |
| harlowja_at_home | agreed | 03:13 |
| fungi | clarkb: jhesketh: which changes? i'm not sussing it from scrollback trivially | 03:14 |
| fungi | oh, 203311 | 03:14 |
| fungi | i'm blind | 03:14 |
| *** jpeeler has quit IRC | 03:15 | |
| *** achanda has quit IRC | 03:17 | |
| *** Sukhdev has joined #openstack-infra | 03:18 | |
| openstackgerrit | Merged openstack-infra/os-loganalyze: Surpress 404 errors from swift https://review.openstack.org/203311 | 03:19 |
| *** Kennan has quit IRC | 03:20 | |
| fungi | jhesketh: ^ | 03:23 |
| *** baoli has quit IRC | 03:23 | |
| *** bharath has joined #openstack-infra | 03:26 | |
| jhesketh | fungi: thanks :-) | 03:29 |
| jhesketh | (sorry went off to make some lunch) | 03:30 |
| fungi | i'm going off to get some sleep | 03:30 |
| *** bharath has quit IRC | 03:30 | |
| jhesketh | seeya | 03:30 |
| jhesketh | thanks for your help :-) | 03:30 |
| openstackgerrit | litong01 proposed openstack-infra/project-config: Add new cloud-monitor repository for OpenStack monitoring service https://review.openstack.org/203259 | 03:31 |
| *** pal has joined #openstack-infra | 03:31 | |
| *** harlowja_at_home has quit IRC | 03:32 | |
| jhesketh | clarkb: error.log and error.log.1 are 17GB and 23GB respectfully | 03:33 |
| jhesketh | the disk has 221GB free space, so it's not an issue | 03:33 |
| *** Sukhdev has quit IRC | 03:33 | |
| jhesketh | should we just let them rotate out normally | 03:33 |
| clarkb | that works for me | 03:34 |
| *** Kennan has joined #openstack-infra | 03:35 | |
| jhesketh | okay then, os-loganalyze is looking good :-) | 03:36 |
| *** sputnik13 has joined #openstack-infra | 03:37 | |
| *** annegentle_ has joined #openstack-infra | 03:41 | |
| *** hdd has joined #openstack-infra | 03:42 | |
| *** annegentle_ has quit IRC | 03:45 | |
| *** hdd has quit IRC | 03:47 | |
| *** tongli has joined #openstack-infra | 03:47 | |
| *** asselin__ has quit IRC | 03:47 | |
| *** mpopow has joined #openstack-infra | 03:49 | |
| openstackgerrit | litong01 proposed openstack-infra/project-config: Add new cloud-monitor repository for OpenStack monitoring service https://review.openstack.org/203259 | 03:57 |
| *** Sukhdev has joined #openstack-infra | 03:58 | |
| *** Sukhdev has quit IRC | 04:03 | |
| *** yamahata has joined #openstack-infra | 04:05 | |
| *** pal has quit IRC | 04:10 | |
| *** bhunter71_2 has quit IRC | 04:11 | |
| *** Sukhdev has joined #openstack-infra | 04:13 | |
| lifeless | oh, has anyone proposed the revert to upper-constraints.txt? | 04:21 |
| lifeless | cause, thats what needs to happen to fix the gate... | 04:22 |
| *** tongli has quit IRC | 04:23 | |
| *** tongli has joined #openstack-infra | 04:24 | |
| *** caphrim007 has joined #openstack-infra | 04:24 | |
| lifeless | ahh https://review.openstack.org/#/c/203295/ | 04:25 |
| lifeless | thats what needs a second +2A | 04:25 |
| lifeless | clarkb: ^ jeblair: ^ | 04:25 |
| *** caphrim007 has quit IRC | 04:26 | |
| *** caphrim007 has joined #openstack-infra | 04:27 | |
| *** caphrim007 has quit IRC | 04:30 | |
| *** tongli has quit IRC | 04:32 | |
| *** tongli has joined #openstack-infra | 04:33 | |
| *** bdemers has quit IRC | 04:33 | |
| *** bdemers has joined #openstack-infra | 04:33 | |
| *** tongli has quit IRC | 04:35 | |
| *** ddieterly has quit IRC | 04:36 | |
| *** changbl has quit IRC | 04:44 | |
| *** Sukhdev has quit IRC | 04:48 | |
| *** caphrim007 has joined #openstack-infra | 04:48 | |
| *** Sukhdev has joined #openstack-infra | 04:54 | |
| *** baoli has joined #openstack-infra | 04:54 | |
| *** kushal has quit IRC | 04:56 | |
| *** FallenPegasus has joined #openstack-infra | 04:58 | |
| *** baoli has quit IRC | 04:59 | |
| *** kushal has joined #openstack-infra | 05:12 | |
| *** bdemers has quit IRC | 05:13 | |
| *** bdemers has joined #openstack-infra | 05:13 | |
| *** bdemers has quit IRC | 05:15 | |
| *** bdemers has joined #openstack-infra | 05:16 | |
| *** Sukhdev has quit IRC | 05:17 | |
| *** Sukhdev has joined #openstack-infra | 05:21 | |
| *** mrmartin has joined #openstack-infra | 05:25 | |
| *** btully has joined #openstack-infra | 05:26 | |
| *** crc32 has quit IRC | 05:27 | |
| *** annegentle_ has joined #openstack-infra | 05:30 | |
| *** bdemers has quit IRC | 05:30 | |
| *** bdemers has joined #openstack-infra | 05:31 | |
| *** flashgordon has quit IRC | 05:32 | |
| *** annegentle_ has quit IRC | 05:34 | |
| *** FallenPegasus has quit IRC | 05:36 | |
| *** ddieterly has joined #openstack-infra | 05:37 | |
| *** bdemers has quit IRC | 05:40 | |
| *** bdemers has joined #openstack-infra | 05:41 | |
| *** mrmartin has quit IRC | 05:41 | |
| *** ddieterly has quit IRC | 05:41 | |
| *** otter768 has quit IRC | 05:46 | |
| *** flashgordon has joined #openstack-infra | 05:48 | |
| *** caphrim007 has quit IRC | 05:49 | |
| *** yamahata has quit IRC | 05:54 | |
| *** j^2 has quit IRC | 05:54 | |
| *** j^2 has joined #openstack-infra | 05:55 | |
| *** bdemers has quit IRC | 05:57 | |
| *** bdemers has joined #openstack-infra | 05:58 | |
| *** tlian has quit IRC | 06:04 | |
| *** sputnik13 has quit IRC | 06:06 | |
| *** sputnik13 has joined #openstack-infra | 06:09 | |
| *** fabo has quit IRC | 06:09 | |
| *** fabo has joined #openstack-infra | 06:17 | |
| *** bdemers has quit IRC | 06:21 | |
| *** SumitNaiksatam has joined #openstack-infra | 06:35 | |
| *** ddieterly has joined #openstack-infra | 06:38 | |
| *** liam__ has joined #openstack-infra | 06:39 | |
| *** achanda has joined #openstack-infra | 06:40 | |
| *** liam__ has quit IRC | 06:41 | |
| *** liam__ has joined #openstack-infra | 06:41 | |
| *** ddieterly has quit IRC | 06:43 | |
| *** liam__ has quit IRC | 06:43 | |
| *** pcaruana has joined #openstack-infra | 06:52 | |
| *** mfalatic has quit IRC | 06:54 | |
| *** nagyz__ has joined #openstack-infra | 06:55 | |
| *** SumitNaiksatam has quit IRC | 06:56 | |
| *** nagyz_ has quit IRC | 06:58 | |
| *** sergmelikyan has joined #openstack-infra | 07:01 | |
| *** SumitNaiksatam has joined #openstack-infra | 07:02 | |
| *** sergmelikyan has quit IRC | 07:11 | |
| *** annegentle_ has joined #openstack-infra | 07:19 | |
| *** unicell1 has quit IRC | 07:19 | |
| *** unicell has joined #openstack-infra | 07:20 | |
| *** Sukhdev has quit IRC | 07:21 | |
| *** annegentle_ has quit IRC | 07:23 | |
| electrical | Morning all | 07:29 |
| *** unicell has quit IRC | 07:39 | |
| *** ddieterly has joined #openstack-infra | 07:39 | |
| *** ddieterly has quit IRC | 07:44 | |
| *** btully has quit IRC | 08:08 | |
| *** ihrachyshka has joined #openstack-infra | 08:23 | |
| *** ihrachyshka has quit IRC | 08:33 | |
| *** achanda has quit IRC | 08:35 | |
| *** e0ne has joined #openstack-infra | 08:36 | |
| *** achanda has joined #openstack-infra | 08:38 | |
| *** e0ne has quit IRC | 08:39 | |
| *** ddieterly has joined #openstack-infra | 08:40 | |
| *** achanda has quit IRC | 08:44 | |
| openstackgerrit | yolanda.robla proposed openstack-infra/system-config: Start using cgit::lb class https://review.openstack.org/181829 | 08:44 |
| *** ddieterly has quit IRC | 08:45 | |
| *** Kennan has quit IRC | 08:49 | |
| openstackgerrit | yolanda.robla proposed openstack-infra/puppet-jeepyb: Separate the fetch remotes log dir and file https://review.openstack.org/192187 | 08:52 |
| *** Kennan has joined #openstack-infra | 08:53 | |
| openstackgerrit | yolanda.robla proposed openstack-infra/puppet-gerrit: New logrotate class in gerrit https://review.openstack.org/201312 | 08:57 |
| openstackgerrit | yolanda.robla proposed openstack-infra/puppet-gerrit: New logrotate class in gerrit https://review.openstack.org/201312 | 08:59 |
| openstackgerrit | yolanda.robla proposed openstack-infra/system-config: Start using gerrit logrotation options https://review.openstack.org/203331 | 09:00 |
| *** annegentle_ has joined #openstack-infra | 09:07 | |
| *** Kiall has quit IRC | 09:10 | |
| *** annegentle_ has quit IRC | 09:12 | |
| *** Kiall has joined #openstack-infra | 09:13 | |
| *** Kennan has quit IRC | 09:17 | |
| *** geoffarnold has joined #openstack-infra | 09:17 | |
| *** heyongli has quit IRC | 09:29 | |
| *** heyongli has joined #openstack-infra | 09:30 | |
| *** Kennan has joined #openstack-infra | 09:30 | |
| *** ihrachyshka has joined #openstack-infra | 09:36 | |
| *** ddieterly has joined #openstack-infra | 09:40 | |
| *** asselin has joined #openstack-infra | 09:44 | |
| *** ddieterly has quit IRC | 09:45 | |
| *** pal has joined #openstack-infra | 09:47 | |
| *** asselin has quit IRC | 09:48 | |
| *** isviridov_away is now known as isviridov | 09:56 | |
| *** geoffarnold has quit IRC | 10:04 | |
| *** geoffarnold has joined #openstack-infra | 10:06 | |
| *** asselin_ has quit IRC | 10:18 | |
| *** kmartin has quit IRC | 10:19 | |
| *** isviridov is now known as isviridov_away | 10:26 | |
| *** Kennan has quit IRC | 10:38 | |
| *** ddieterly has joined #openstack-infra | 10:41 | |
| *** ddieterly has quit IRC | 10:46 | |
| *** marzif__ has joined #openstack-infra | 10:50 | |
| *** Kennan has joined #openstack-infra | 10:51 | |
| *** annegentle_ has joined #openstack-infra | 10:56 | |
| *** annegentle_ has quit IRC | 11:01 | |
| *** rossella_s has joined #openstack-infra | 11:02 | |
| openstackgerrit | Morgan Fainberg proposed openstack/requirements: ldappool is only python-2.7 https://review.openstack.org/203336 | 11:04 |
| openstackgerrit | Morgan Fainberg proposed openstack/requirements: Add ldap3 to global-requirements https://review.openstack.org/176556 | 11:04 |
| *** ddieterly has joined #openstack-infra | 11:07 | |
| *** pal has quit IRC | 11:09 | |
| *** _nadya_ has joined #openstack-infra | 11:12 | |
| *** pal has joined #openstack-infra | 11:14 | |
| *** rguillebert has joined #openstack-infra | 11:18 | |
| openstackgerrit | yolanda.robla proposed openstack-infra/puppet-jeepyb: Separate the fetch remotes log dir and file https://review.openstack.org/192187 | 11:20 |
| *** _nadya_ has quit IRC | 11:26 | |
| *** _nadya_ has joined #openstack-infra | 11:27 | |
| *** _nadya_ has quit IRC | 11:31 | |
| openstackgerrit | yolanda.robla proposed openstack-infra/puppet-jeepyb: Separate the fetch remotes log dir and file https://review.openstack.org/192187 | 11:35 |
| *** bdemers has joined #openstack-infra | 11:43 | |
| *** dizquierdo has joined #openstack-infra | 11:50 | |
| *** geoffarnold has quit IRC | 12:18 | |
| *** HeOS has quit IRC | 12:19 | |
| *** dims_ has joined #openstack-infra | 12:20 | |
| *** ildikov has joined #openstack-infra | 12:22 | |
| *** dizquierdo has quit IRC | 12:29 | |
| *** pal has quit IRC | 12:31 | |
| *** ruagair has quit IRC | 12:34 | |
| *** mriedem has joined #openstack-infra | 12:40 | |
| *** pal has joined #openstack-infra | 12:43 | |
| *** EricGonczer_ has joined #openstack-infra | 12:45 | |
| *** annegentle_ has joined #openstack-infra | 12:45 | |
| *** annegentle_ has quit IRC | 12:50 | |
| *** Kiall has quit IRC | 12:50 | |
| *** Kiall has joined #openstack-infra | 12:50 | |
| *** ruagair has joined #openstack-infra | 12:51 | |
| *** Kiall has quit IRC | 13:00 | |
| *** pal has quit IRC | 13:00 | |
| *** Kiall has joined #openstack-infra | 13:00 | |
| *** pal has joined #openstack-infra | 13:00 | |
| *** heyongli has quit IRC | 13:02 | |
| *** heyongli has joined #openstack-infra | 13:03 | |
| *** mhu has quit IRC | 13:05 | |
| *** miand has joined #openstack-infra | 13:07 | |
| *** mhu has joined #openstack-infra | 13:08 | |
| *** chlong has joined #openstack-infra | 13:09 | |
| *** EricGonczer_ has quit IRC | 13:09 | |
| greghaynes | ianw: I think https://review.openstack.org/#/c/186620/7 requires a rebase maybe? | 13:11 |
| greghaynes | its pretty unclear what is going on, but gerrit is definitely not happy with it | 13:12 |
| greghaynes | seems like maybe a jgit / zuul git not doing the same thing deal | 13:12 |
| *** michchap has quit IRC | 13:24 | |
| *** michchap has joined #openstack-infra | 13:32 | |
| *** nagyz__ has quit IRC | 13:38 | |
| *** e0ne has joined #openstack-infra | 13:40 | |
| *** e0ne has quit IRC | 13:45 | |
| openstackgerrit | Ihar Hrachyshka proposed openstack-infra/project-config: neutron: enable qos ml2 extension driver for feature/qos https://review.openstack.org/203351 | 13:46 |
| *** ihrachyshka has quit IRC | 13:58 | |
| *** annegentle_ has joined #openstack-infra | 14:01 | |
| *** e0ne has joined #openstack-infra | 14:04 | |
| *** e0ne has quit IRC | 14:07 | |
| greghaynes | mriedem: re: your ML post, are you sure that is the fial causing this? there is also http://logs.openstack.org/95/203295/1/check/gate-requirements-integration-dsvm/10073dd/console.html#_2015-07-18_13_16_32_832 later on... | 14:07 |
| greghaynes | mtreinish: which might just be messing uuid-deb | 14:08 |
| greghaynes | er, uuid-dev | 14:08 |
| *** btully has joined #openstack-infra | 14:10 | |
| *** e0ne has joined #openstack-infra | 14:10 | |
| mriedem | greghaynes: ah yeah that's probably it | 14:10 |
| mriedem | so, i guess i'll try adding that to devstack's list of files for debs | 14:10 |
| greghaynes | yep | 14:11 |
| mriedem | i'm not sure why this would be a new failure in the last 24 hours | 14:11 |
| greghaynes | looks like it was also causing sqlite3's wheel build to fail which proably doesnt matter | 14:11 |
| greghaynes | uhhh, me either, could be new release on their part or could be something wierd with new images | 14:11 |
| mriedem | yeah, doesn't matter at this point i guess | 14:12 |
| *** e0ne has quit IRC | 14:12 | |
| mriedem | opened https://bugs.launchpad.net/devstack/+bug/1475880 | 14:13 |
| openstack | Launchpad bug 1475880 in devstack "gate-requirements-integration-dsvm fails due to missing uuid-dev package" [Undecided,Confirmed] - Assigned to Matt Riedemann (mriedem) | 14:13 |
| mriedem | will push a change | 14:13 |
| mriedem | le sigh | 14:13 |
| mriedem | it's a good thing i've got dora the explorer on in the background to keep my spirits up | 14:13 |
| mriedem | "you can do it!" | 14:13 |
| greghaynes | haha | 14:13 |
| *** btully has quit IRC | 14:14 | |
| *** e0ne has joined #openstack-infra | 14:17 | |
| *** hrubi has joined #openstack-infra | 14:20 | |
| *** FallenPegasus has joined #openstack-infra | 14:21 | |
| *** FallenPegasus has quit IRC | 14:22 | |
| *** FallenPegasus has joined #openstack-infra | 14:26 | |
| openstackgerrit | Merged openstack-infra/glean: Refuse to replace pre-existing interface configs. https://review.openstack.org/202984 | 14:27 |
| *** FallenPegasus has quit IRC | 14:28 | |
| *** annegentle_ has quit IRC | 14:30 | |
| greghaynes | Ng: mordred ^ | 14:31 |
| openstackgerrit | Matt Riedemann proposed openstack/requirements: Exclude glance_store 0.7.0 due to gate blocking regression https://review.openstack.org/203295 | 14:32 |
| mriedem | fungi: clarkb: ^ added the depends-on for the devstack change | 14:32 |
| fungi | mriedem: dims_: anybody happen to know how we got in a state of oslo.messaging needing uuid-dev and swig in devstack? | 14:33 |
| *** amotoki has joined #openstack-infra | 14:33 | |
| mriedem | fungi: https://review.openstack.org/#/c/186671/ | 14:34 |
| fungi | i mean, i get that it's needed to be able to build proton, just | 14:34 |
| mriedem | oslo.messaging has amqp as an optional backend, | 14:34 |
| mriedem | which is qpid-proton,] | 14:34 |
| mriedem | which requires those other things to build | 14:34 |
| fungi | and enabling it passed devstack jobs | 14:35 |
| mriedem | i'm looking to see if there is a way to just restrict this to the gate-requirements-integration-dsvm | 14:35 |
| *** annegentle_ has joined #openstack-infra | 14:35 | |
| mriedem | since i'm guessing sdague won't want those packages listed in devstack | 14:35 |
| fungi | just wondering how oslo.messaging with 186671 was installed in devstack even though uuid-dev and swig weren't present | 14:36 |
| fungi | looking at the job logs now to see if i can figure it out | 14:36 |
| fungi | i suppose it's only tickled by the requirements integration job | 14:37 |
| mriedem | yeah devstack doesn't install qpid-proton | 14:37 |
| mriedem | that'd only be via the amqp job for oslo.messaging i guess | 14:37 |
| mriedem | with that post hook | 14:37 |
| *** e0ne has quit IRC | 14:39 | |
| mriedem | so i'm guessing that the packages should actually be installed here: https://github.com/openstack/requirements/blob/master/tools/integration.sh | 14:40 |
| mriedem | that's what's called from the project definition in project-config | 14:40 |
| mriedem | and that would keep these things out of devstack if you're not using amqp | 14:40 |
| fungi | right, requirements just added (i think it merged?) a bindep other-requirements.txt file to make those easier to manage | 14:41 |
| mriedem | https://github.com/openstack/requirements/blob/master/other-requirements.txt | 14:41 |
| mriedem | yup | 14:41 |
| mriedem | nice | 14:41 |
| mriedem | however | 14:42 |
| mriedem | that already has uuid-dev and swig listed | 14:42 |
| *** e0ne has joined #openstack-infra | 14:44 | |
| fungi | from the log, it's also failing to find sqlite3.h | 14:44 |
| *** dizquierdo has joined #openstack-infra | 14:44 | |
| fungi | 2015-07-18 13:07:59.643 | /opt/stack/new/requirements/tools/integration.sh: line | 14:46 |
| fungi | 29: /usr/bindep-env/bin/bindep: No such file or directory | 14:46 |
| fungi | built on devstack-trusty rax-iad so it should have it installed already | 14:47 |
| *** gyee has joined #openstack-infra | 14:47 | |
| mriedem | yeah https://review.openstack.org/#/c/199782/ merged yesterday | 14:48 |
| *** superdan is now known as dansmith | 14:48 | |
| mriedem | so that's what changed in the last 24 hours to break | 14:48 |
| mriedem | gdi | 14:48 |
| fungi | confirmed, for some reason /usr/bindep-env/bin/bindep is not on that image. checking the image update logs now to see why | 14:48 |
| *** hdd has joined #openstack-infra | 14:49 | |
| mriedem | fungi: can we just revert this for now | 14:49 |
| mriedem | it's saturday and i'm 3 blockers deep in the gate by now | 14:49 |
| *** anthonyper has quit IRC | 14:49 | |
| *** anthonyper has joined #openstack-infra | 14:49 | |
| openstackgerrit | Matt Riedemann proposed openstack/requirements: Revert "Move integration test dependencies into bindep" https://review.openstack.org/203355 | 14:50 |
| openstackgerrit | Matt Riedemann proposed openstack/requirements: Exclude glance_store 0.7.0 due to gate blocking regression https://review.openstack.org/203295 | 14:50 |
| *** puranamr has joined #openstack-infra | 14:50 | |
| *** e0ne has quit IRC | 14:53 | |
| fungi | aww crap | 14:53 |
| fungi | i see the problem | 14:53 |
| fungi | we're not uploading dib-built devstack-trusty images to rackspace, just hpcloud | 14:54 |
| fungi | i thought we already were | 14:54 |
| fungi | we'll need to revert 199782 for now | 14:54 |
| *** gyee has quit IRC | 14:54 | |
| fungi | greghaynes: mordred: ^ do you know what's left to do before we start using dib-built devstack-trusty images in rackspace? or do we need to move all the devstack-based jobs to ubuntu-trusty nodes instead? | 14:55 |
| anteaya | yesterday sometime you were talking about reducing the amount of apache logs we retain, fungi. clarkb mentioned I had a patch on it, yolanda has taken it over, here it is, your thoughts welcome: https://review.openstack.org/#/c/201312/10 | 14:56 |
| *** e0ne has joined #openstack-infra | 14:57 | |
| *** asselin has joined #openstack-infra | 14:57 | |
| mriedem | fungi: yup, got the revert posted https://review.openstack.org/#/c/203355/ | 14:57 |
| greghaynes | fungi: correct, no dib is actually being used in rax yet. There was some error handling that went into shade that has been released, I think there might also be some more packages missing for multinode... | 14:58 |
| greghaynes | I wonder if the job to run ubuntu-trusty multinode merged | 14:58 |
| greghaynes | https://review.openstack.org/#/c/202093/ | 14:59 |
| *** hdd has quit IRC | 14:59 | |
| greghaynes | fungi: ^ we need to merge that and then cycle on possibly more missing packages in the images | 14:59 |
| openstackgerrit | Jeremy Stanley proposed openstack/requirements: Revert "Move integration test dependencies into bindep" https://review.openstack.org/203357 | 15:00 |
| openstackgerrit | Jeremy Stanley proposed openstack/requirements: Move integration test dependencies into bindep https://review.openstack.org/203358 | 15:00 |
| fungi | mriedem: lifeless: ^ i'll wip the second one for now | 15:00 |
| anteaya | greghaynes: do you understand the failure in the log for that patch? | 15:00 |
| greghaynes | anteaya: just did a recheck, it was a git cloning fail | 15:00 |
| anteaya | A TLS packet with unexpected length was received. | 15:01 |
| anteaya | okay | 15:01 |
| mriedem | fungi: no arguments from me except i have the glance_store block change dependent on my revert | 15:03 |
| mriedem | fungi: so either we need to drop mine and point at yours, or drop yours and probably copy the commit message since it's better | 15:03 |
| anteaya | greghaynes: +2 from me assuming you can get jenkins in a happy place | 15:03 |
| fungi | mriedem: oh, i missed you saying you were working on proposing a revert. sorry about that | 15:03 |
| mriedem | fungi: should i just steal your commit message? | 15:04 |
| * fungi will stop duplicating effort now | 15:04 | |
| fungi | feel free | 15:04 |
| openstackgerrit | Matt Riedemann proposed openstack/requirements: Revert "Move integration test dependencies into bindep" https://review.openstack.org/203355 | 15:04 |
| mriedem | done | 15:04 |
| mriedem | thanks | 15:04 |
| fungi | np | 15:05 |
| *** mhu has quit IRC | 15:05 | |
| Ng | greghaynes: woot! Thanks :) | 15:05 |
| Ng | greghaynes: fancy looking at the one that depends on it? https://review.openstack.org/#/c/202250/ | 15:06 |
| *** ddieterly has quit IRC | 15:07 | |
| *** mhu has joined #openstack-infra | 15:07 | |
| openstackgerrit | Jeremy Stanley proposed openstack/requirements: Move integration test dependencies into bindep https://review.openstack.org/203358 | 15:08 |
| *** pal has quit IRC | 15:08 | |
| fungi | mriedem: okay, rebased the unrevert on top of your revert | 15:08 |
| mriedem | yup, see that | 15:09 |
| mriedem | might want to wip your unrevert again | 15:09 |
| mriedem | fungi: also, want to re-approve the glance_store block? https://review.openstack.org/#/c/203295/ | 15:09 |
| *** puranamr has quit IRC | 15:09 | |
| fungi | yep | 15:10 |
| fungi | hrm, it has a depends-on to another requirements change rather than being rebased onto it | 15:11 |
| *** e0ne has quit IRC | 15:11 | |
| fungi | i guess that's okay | 15:11 |
| mriedem | yeah guess i didn't think about that | 15:12 |
| fungi | i've bumped them straight to the gate pipeline to get this knocked out a little sooner | 15:13 |
| *** dizquierdo has quit IRC | 15:13 | |
| *** changbl has joined #openstack-infra | 15:14 | |
| *** e0ne has joined #openstack-infra | 15:15 | |
| *** e0ne has quit IRC | 15:17 | |
| dims_ | yay fungi thanks | 15:19 |
| *** e0ne has joined #openstack-infra | 15:20 | |
| *** e0ne has quit IRC | 15:23 | |
| fungi | well, i feel fairly responsible for the situation since i let that change through not realizing we weren't dib'ing devstack images in rax yet | 15:24 |
| *** btully has joined #openstack-infra | 15:24 | |
| *** e0ne has joined #openstack-infra | 15:26 | |
| greghaynes | fungi: https://review.openstack.org/#/c/202093/ passed now | 15:27 |
| greghaynes | and once that goes in I can cycle on stuff needed to move over to dib in rax | 15:27 |
| *** e0ne has quit IRC | 15:29 | |
| *** e0ne has joined #openstack-infra | 15:32 | |
| fungi | +3 | 15:34 |
| *** e0ne has quit IRC | 15:35 | |
| openstackgerrit | Merged openstack-infra/project-config: Add experimental multi-node job to use dib images https://review.openstack.org/202093 | 15:37 |
| *** e0ne has joined #openstack-infra | 15:40 | |
| *** e0ne has quit IRC | 15:42 | |
| *** sputnik13 has quit IRC | 15:43 | |
| *** tlian has joined #openstack-infra | 15:46 | |
| *** e0ne has joined #openstack-infra | 15:46 | |
| *** mwagner_lap has quit IRC | 15:47 | |
| jeblair | fungi: i see things are happening. if i wanted to restart zuul and/or gerrit, what should i ensure is stabilized before i do so? | 15:50 |
| *** e0ne has quit IRC | 15:50 | |
| jeblair | fungi: i'm guessing 355 and 295 landing are the most important things | 15:51 |
| fungi | jeblair: letting the two requirements changes in te gate merge first would be awesome | 15:51 |
| fungi | yep, thanks! | 15:51 |
| jeblair | cool, i think i'm caught up (after reading their commit msgs) | 15:52 |
| *** e0ne has joined #openstack-infra | 15:54 | |
| *** ildikov has quit IRC | 15:55 | |
| *** _fortis has quit IRC | 15:57 | |
| *** e0ne has quit IRC | 15:58 | |
| *** _fortis has joined #openstack-infra | 15:58 | |
| *** ig0r_ has joined #openstack-infra | 16:01 | |
| *** dizquierdo has joined #openstack-infra | 16:01 | |
| *** e0ne has joined #openstack-infra | 16:02 | |
| openstackgerrit | Merged openstack/requirements: Revert "Move integration test dependencies into bindep" https://review.openstack.org/203355 | 16:03 |
| *** _nadya_ has joined #openstack-infra | 16:03 | |
| openstackgerrit | Somay Jain proposed openstack-infra/jenkins-job-builder: Adding support for monitoring files using FSTrigger Plugin https://review.openstack.org/203364 | 16:04 |
| *** ig0r_ has quit IRC | 16:05 | |
| *** miand has quit IRC | 16:07 | |
| *** sdake has joined #openstack-infra | 16:07 | |
| *** ddieterly has joined #openstack-infra | 16:07 | |
| *** ddieterly has quit IRC | 16:12 | |
| *** e0ne has quit IRC | 16:12 | |
| *** mhu has quit IRC | 16:13 | |
| *** Somay has joined #openstack-infra | 16:15 | |
| *** e0ne has joined #openstack-infra | 16:16 | |
| *** pcrews__ has quit IRC | 16:17 | |
| *** e0ne has quit IRC | 16:19 | |
| *** mhu has joined #openstack-infra | 16:19 | |
| *** puranamr has joined #openstack-infra | 16:20 | |
| *** pcrews has joined #openstack-infra | 16:20 | |
| *** FallenPegasus has joined #openstack-infra | 16:21 | |
| *** armax has quit IRC | 16:21 | |
| *** FallenPe_ has joined #openstack-infra | 16:21 | |
| *** armax has joined #openstack-infra | 16:22 | |
| *** e0ne has joined #openstack-infra | 16:23 | |
| *** FallenPegasus has quit IRC | 16:25 | |
| openstackgerrit | Somay Jain proposed openstack-infra/jenkins-job-builder: Adding support for IvyTrigger Plugin https://review.openstack.org/200491 | 16:25 |
| *** annegentle_ has quit IRC | 16:25 | |
| *** sdake_ has joined #openstack-infra | 16:26 | |
| *** _nadya_ has quit IRC | 16:27 | |
| *** _nadya_ has joined #openstack-infra | 16:28 | |
| *** _nadya_ has quit IRC | 16:28 | |
| *** sdake has quit IRC | 16:29 | |
| *** _nadya_ has joined #openstack-infra | 16:29 | |
| *** e0ne has quit IRC | 16:31 | |
| *** Somay has quit IRC | 16:32 | |
| *** puranamr has quit IRC | 16:33 | |
| *** e0ne has joined #openstack-infra | 16:35 | |
| *** e0ne has quit IRC | 16:37 | |
| *** _nadya_ has quit IRC | 16:39 | |
| *** shashank_hegde has joined #openstack-infra | 16:40 | |
| *** _nadya_ has joined #openstack-infra | 16:40 | |
| *** e0ne has joined #openstack-infra | 16:41 | |
| *** armax has quit IRC | 16:42 | |
| *** jsavak has joined #openstack-infra | 16:44 | |
| *** _nadya_ has quit IRC | 16:44 | |
| *** e0ne has quit IRC | 16:46 | |
| *** jsavak has quit IRC | 16:48 | |
| *** jsavak has joined #openstack-infra | 16:48 | |
| *** dims_ has quit IRC | 16:49 | |
| *** e0ne has joined #openstack-infra | 16:50 | |
| jeblair | fungi: ha, i just spent the last 30 minutes trying to figure out why the builds for 203295 restarted -- i thought there was something wrong with the severed head case (the case of the severed head?) where every change behind it might be restarted when it got to the top. but that's not the case -- it works just fine... | 16:51 |
| *** e0ne has quit IRC | 16:53 | |
| *** tlian has quit IRC | 16:53 | |
| jeblair | fungi: i eventually realized it's because a reconfiguration happened right around the time that the change ahead merged; so actually it was restarted because it rebuilt the queue after the reconfiguration and it was different than before (possibly still because of the severed head, but that makes it a much smaller edge case) | 16:54 |
| openstackgerrit | Merged openstack/requirements: Exclude glance_store 0.7.0 due to gate blocking regression https://review.openstack.org/203295 | 16:55 |
| jeblair | yay | 16:55 |
| *** FallenPe_ has quit IRC | 16:55 | |
| *** e0ne has joined #openstack-infra | 16:57 | |
| jeblair | restarting gerrit | 17:00 |
| *** s0nea has quit IRC | 17:02 | |
| *** amotoki has quit IRC | 17:04 | |
| *** e0ne has quit IRC | 17:05 | |
| *** s0nea has joined #openstack-infra | 17:06 | |
| *** ddieterly has joined #openstack-infra | 17:08 | |
| *** marekd has joined #openstack-infra | 17:08 | |
| *** e0ne has joined #openstack-infra | 17:10 | |
| jeblair | restarting zuul | 17:10 |
| *** jamesmcarthur has joined #openstack-infra | 17:11 | |
| *** e0ne has quit IRC | 17:11 | |
| *** ddieterly has quit IRC | 17:13 | |
| fungi | heh, yep, interesting side effect | 17:13 |
| *** pal has joined #openstack-infra | 17:15 | |
| *** e0ne has joined #openstack-infra | 17:16 | |
| jeblair | fungi: i've manually configured gerrit to replicate to afs (via ssh to afstest.o.o for now, just for testing). i note that it is in fact creating the git repositories without the need for manage-projects. | 17:18 |
| *** e0ne has quit IRC | 17:18 | |
| jeblair | fungi: (however, i did pre-create the 'org' directories (eg 'openstack'); i don't know if gerrit can create those on its own) | 17:18 |
| openstackgerrit | Paul Belanger proposed openstack/diskimage-builder: Test both fedora 21 and fedora 22 https://review.openstack.org/203247 | 17:18 |
| openstackgerrit | Paul Belanger proposed openstack/diskimage-builder: Update default Fedora to 22 https://review.openstack.org/185843 | 17:18 |
| openstackgerrit | Paul Belanger proposed openstack/diskimage-builder: Cleanup yum downloading https://review.openstack.org/186620 | 17:18 |
| fungi | jeblair: oh, nice! | 17:19 |
| jeblair | i'll see if i can answer that question next | 17:19 |
| jeblair | it would be cool not to have to run m-p on the git farm | 17:19 |
| fungi | in theory, once it stops replicating, you could just delete stackforge-attic and then force replication for one of the repos under it again | 17:19 |
| jeblair | good idea | 17:20 |
| fungi | smallest org by commit count, probably | 17:20 |
| fungi | btw, i'm mostly afk back and forth with boxes most of the day | 17:21 |
| jeblair | fungi: lift with your legs! | 17:21 |
| jeblair | i mean, you may also want to use arms and hands, etc. | 17:21 |
| *** e0ne has joined #openstack-infra | 17:22 | |
| *** jsavak has quit IRC | 17:22 | |
| *** jsavak has joined #openstack-infra | 17:22 | |
| * fungi lifts with tools | 17:23 | |
| jeblair | oh. also, i should set the quota for the git volume higher than 5M. | 17:23 |
| fungi | oh, yeah that's probably a wise precaution ;) | 17:24 |
| *** e0ne has quit IRC | 17:26 | |
| *** ddieterly has joined #openstack-infra | 17:26 | |
| *** jsavak has quit IRC | 17:28 | |
| jeblair | fungi, clarkb, mordred, pleia2: i've confirmed that gerrit replication over ssh will both create git repos and intermediate directories (eg 'openstack/'). so i think we can look into removing manage-projects from the git servers and simplifying the ansible runs | 17:29 |
| *** zz_dimtruck is now known as dimtruck | 17:29 | |
| *** e0ne has joined #openstack-infra | 17:30 | |
| fungi | that will be quite awesomemaking | 17:31 |
| clarkb | ooh | 17:32 |
| jeblair | and those of you with afs clients can watch /afs/openstack.org/mirror/git being populated right now if you want :) | 17:33 |
| clarkb | is that replicating "locally"? | 17:33 |
| jeblair | clarkb: no, over ssh | 17:34 |
| jeblair | so we can try stuff out without futzing with the client on gerrit | 17:34 |
| *** harlowja_at_home has joined #openstack-infra | 17:35 | |
| *** jamesmcarthur has quit IRC | 17:35 | |
| fungi | and saves us needing to kexec wrap the gerrit jvm (for now at least, we can always decide to try that down the road) | 17:35 |
| jeblair | yep | 17:35 |
| jeblair | i also just gave afstest an ip acl to write to the directory to make that part easier for testing too | 17:36 |
| fungi | i wonder though whether the afs max path length is going to cause issues | 17:36 |
| fungi | i guess for bare repos probably not | 17:36 |
| *** e0ne has quit IRC | 17:38 | |
| fungi | the answer, for anyone curious, is that the longest path rooted at /var/lib/git on our git servers is 126 characters | 17:41 |
| *** dims_ has joined #openstack-infra | 17:41 | |
| *** e0ne has joined #openstack-infra | 17:41 | |
| *** marekd has quit IRC | 17:42 | |
| *** marekd has joined #openstack-infra | 17:42 | |
| jeblair | is there a path length limit? i didn't think so, though there are filename limits (and limits for symlink targets) | 17:43 |
| jeblair | hrm. for some reason the file 'refs/heads/master' is not being replicated | 17:43 |
| jeblair | and it's important | 17:43 |
| fungi | yep, i just went hunting, apparently it's that the number of files you can fit in a directory decreases the longer the filenames are | 17:44 |
| fungi | not the max path length | 17:44 |
| jeblair | yep | 17:44 |
| *** shashank_hegde has quit IRC | 17:44 | |
| *** ildikov has joined #openstack-infra | 17:44 | |
| fungi | 64k slots and every 16 bytes or fraction thereof in a filename takes one slot | 17:45 |
| jeblair | i wonder why the heads are not being replicated... and i wonder if that's something that we get as a side effect of m-p... | 17:45 |
| dims_ | fungi: mriedem: i'll cut the glance_store release to unblock gates | 17:46 |
| fungi | jeblair: looking at the replication config documentation, it looks like pushing refs/heads/* and refs/tags/* is optional | 17:47 |
| jeblair | i copied the cgit configuration... | 17:47 |
| fungi | yeah, that's seriously strange | 17:47 |
| fungi | because it seems to be working there | 17:47 |
| mriedem | dims_: it actually looks like glance's requirements pull glance_store from source rather than pypi | 17:48 |
| jeblair | though that is slightly complicated by the fact that we pack refs on cgit... | 17:48 |
| mriedem | dims_: so just getting my fix into glance_store might have fixed things | 17:48 |
| *** enikanorov_ has quit IRC | 17:48 | |
| mriedem | dims_: https://github.com/openstack/glance/blob/master/requirements.txt#L63 | 17:48 |
| jeblair | but the gerrit repo itself doesn't, so i'd expect it to look like that | 17:48 |
| *** enikanorov has joined #openstack-infra | 17:49 | |
| jeblair | fungi: oh weird | 17:49 |
| jeblair | fungi: when replicate a second time, it pushes it | 17:50 |
| fungi | that's pure strangeness | 17:50 |
| jeblair | so i wonder if there's a malfunction during the initial creation | 17:50 |
| fungi | sounds like a gerrit bug | 17:50 |
| fungi | yeah, maybe it gets interrupted | 17:50 |
| fungi | or interrupts itself anyway | 17:50 |
| jeblair | we should probably verify that's a gerrit replication thing vs an afs thing | 17:51 |
| *** achanda has joined #openstack-infra | 17:51 | |
| * jeblair kicks off the fourth full replication pass of the morning | 17:52 | |
| jeblair | (and indeed, i'm now watching refs/heads/master show up one repo at a time) | 17:52 |
| jeblair | this works now: git --git-dir /afs/openstack.org/mirror/git/openstack-infra/devstack-gate.git log | 17:54 |
| openstackgerrit | Matt Riedemann proposed openstack/requirements: Bump glance_store to 0.7.1 for py34 compatibility https://review.openstack.org/203372 | 17:54 |
| *** e0ne has quit IRC | 17:55 | |
| fungi | slick! | 17:56 |
| *** HeOS has joined #openstack-infra | 17:58 | |
| *** e0ne has joined #openstack-infra | 17:59 | |
| *** nelsnelson has joined #openstack-infra | 18:00 | |
| *** e0ne has quit IRC | 18:02 | |
| *** thedodd has joined #openstack-infra | 18:03 | |
| *** puranamr has joined #openstack-infra | 18:04 | |
| *** amotoki has joined #openstack-infra | 18:04 | |
| dims_ | mriedem: ack. i've cut the 0.7.1 anyways :) | 18:05 |
| *** e0ne has joined #openstack-infra | 18:06 | |
| *** pal has quit IRC | 18:06 | |
| *** puranamr has quit IRC | 18:06 | |
| *** e0ne has quit IRC | 18:08 | |
| *** puranamr has joined #openstack-infra | 18:08 | |
| *** amotoki has quit IRC | 18:09 | |
| *** amotoki has joined #openstack-infra | 18:11 | |
| *** e0ne has joined #openstack-infra | 18:11 | |
| *** puranamr has quit IRC | 18:15 | |
| *** e0ne has quit IRC | 18:17 | |
| *** harlowja_at_home has quit IRC | 18:19 | |
| *** ddieterl_ has joined #openstack-infra | 18:20 | |
| *** ddieterly has quit IRC | 18:20 | |
| *** e0ne has joined #openstack-infra | 18:20 | |
| kragniz | mriedem: ah yes, 0.7.0 was mostly so we could fix that up | 18:22 |
| *** e0ne has quit IRC | 18:23 | |
| mriedem | kragniz: the py34 stuff? | 18:23 |
| kragniz | yeah | 18:23 |
| mriedem | anyway, dims_ kragniz fungi we're golden now https://jenkins06.openstack.org/job/gate-tempest-dsvm-full-ceph/597/console | 18:23 |
| mriedem | thanks! | 18:23 |
| *** geoffarnold has joined #openstack-infra | 18:24 | |
| kragniz | mriedem: good job! | 18:24 |
| mriedem | time to celebrate with a flav-or-ice | 18:25 |
| mriedem | *flav-r-ice? | 18:25 |
| *** btully has quit IRC | 18:27 | |
| *** e0ne has joined #openstack-infra | 18:27 | |
| *** e0ne has quit IRC | 18:28 | |
| *** _nadya_ has joined #openstack-infra | 18:29 | |
| fungi | mriedem: thanks for staying on top of that! | 18:30 |
| jeblair | restarting gerrit again | 18:31 |
| jeblair | i've just learned about sysctl kernel.yama.protected_nonaccess_hardlinks | 18:31 |
| *** kdas__ has joined #openstack-infra | 18:32 | |
| *** achanda has quit IRC | 18:33 | |
| *** geoffarnold has quit IRC | 18:34 | |
| *** geoffarnold has joined #openstack-infra | 18:34 | |
| *** dizquierdo has quit IRC | 18:35 | |
| *** jsavak has joined #openstack-infra | 18:35 | |
| *** kushal has quit IRC | 18:35 | |
| *** shashank_hegde has joined #openstack-infra | 18:38 | |
| *** jsavak has quit IRC | 18:39 | |
| *** achanda has joined #openstack-infra | 18:41 | |
| *** tongli has joined #openstack-infra | 18:43 | |
| pabelanger | Well, that explains why nodepool image-build is failing, looks like a bug | 18:44 |
| pabelanger | and lack of test coverage | 18:44 |
| *** EricGonczer_ has joined #openstack-infra | 18:44 | |
| fungi | apparently fs.protected_hardlinks in later kernels | 18:45 |
| *** changbl has quit IRC | 18:48 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/nodepool: [WIP] Add test_dib_image_build test https://review.openstack.org/203375 | 18:49 |
| *** geoffarnold has quit IRC | 18:52 | |
| *** maishsk has joined #openstack-infra | 18:57 | |
| jeblair | fungi, clarkb, mordred, pleia2: so, er, i think i'm actually confirming that gerrit may or may not create missing repositories based on some criteria that i have not determined. it created 392 of our 728 repos. | 18:57 |
| jeblair | i suspect a bug in the replication plugin; perhaps one that only shows up if you have >1 replication target | 19:00 |
| jeblair | at any rate, my hopes of being able to avoid m-p are being dashed. | 19:00 |
| pabelanger | Yup, nodepool image-build is broken. It expects an image-type however, no way to set it from diskimages section in config | 19:08 |
| *** HeOS has quit IRC | 19:09 | |
| openstackgerrit | litong01 proposed openstack-infra/project-config: Add new monitor repository for OpenStack monitoring service https://review.openstack.org/203259 | 19:10 |
| openstackgerrit | litong01 proposed openstack-infra/project-config: Add new monitor repository for OpenStack monitoring service https://review.openstack.org/203259 | 19:10 |
| *** marekd has quit IRC | 19:12 | |
| clarkb | pabelanger I used it jusya couple days ago and it worked | 19:12 |
| *** amotoki has quit IRC | 19:12 | |
| clarkb | its jus image-build diskimage-name | 19:12 |
| clarkb | made vhd and qcow2 for since nodepool.yaml was set up for both | 19:13 |
| clarkb | should do qcow2 by default | 19:13 |
| *** EricGonczer_ has quit IRC | 19:14 | |
| pabelanger | 2015-07-18 18:30:29,593 INFO nodepool.DiskImageBuilderThread: Running disk-image-create -x -t --no-tmpfs -o /opt/nodepool_dib/devstack-fedora22-dib-1437244229 fedora | 19:14 |
| pabelanger | clarkb: ^ node the empty value after -t | 19:14 |
| clarkb | it should default to qcow2 hrm | 19:14 |
| clarkb | maybe mine worjed because we are explicit | 19:15 |
| pabelanger | that is my using: $ nodepool --debug image-build devstack-fedora22-dib | 19:15 |
| pabelanger | might you | 19:15 |
| pabelanger | I don't have devstack-fedora22-dib declared in a provider yet | 19:15 |
| pabelanger | so, I think that is the issue | 19:15 |
| clarkb | oh ya thats it | 19:15 |
| pabelanger | I just added it under the diskimages section | 19:15 |
| clarkb | nodepool isnt going to find any provider info and be confused since that determines what image type to use | 19:16 |
| *** btully has joined #openstack-infra | 19:16 | |
| clarkb | its required to be in a provider but may have min ready 0 | 19:16 |
| *** puranamr has joined #openstack-infra | 19:17 | |
| *** esmute has joined #openstack-infra | 19:17 | |
| *** EricGonczer_ has joined #openstack-infra | 19:18 | |
| openstackgerrit | Ian Cordasco proposed openstack/requirements: Add keystoneauth to projects.txt https://review.openstack.org/203256 | 19:18 |
| pabelanger | clarkb: added. Lets see what happens now | 19:18 |
| *** armax has joined #openstack-infra | 19:18 | |
| *** jsavak has joined #openstack-infra | 19:20 | |
| *** FallenPegasus has joined #openstack-infra | 19:20 | |
| pabelanger | clarkb: ya, better now | 19:21 |
| pabelanger | either way, I have a patchset up to add test coverage to the command, since it is missing | 19:21 |
| pabelanger | will work on it this week and see if I can clean it up | 19:22 |
| *** armax has quit IRC | 19:22 | |
| *** shashank_hegde has quit IRC | 19:23 | |
| pabelanger | now to fix puppet for fedora 22 | 19:23 |
| *** Somay has joined #openstack-infra | 19:24 | |
| *** flashgordon has quit IRC | 19:25 | |
| openstackgerrit | litong01 proposed openstack-infra/project-config: Add new monitor repository for OpenStack monitoring service https://review.openstack.org/203259 | 19:25 |
| clarkb | fwiw you can also just run dib directly if iterating on a new image. I find that to be easier as I can jump i to the image if needes wjthout nodepool deleting it | 19:26 |
| *** marekd has joined #openstack-infra | 19:27 | |
| *** mriedem has quit IRC | 19:29 | |
| *** shashank_hegde has joined #openstack-infra | 19:30 | |
| *** tongli has quit IRC | 19:32 | |
| *** puranamr has quit IRC | 19:32 | |
| pabelanger | ack | 19:34 |
| pabelanger | Anyways, calling it for now. Ran into https://review.openstack.org/#/c/202884/ for puppet-4. | 19:34 |
| pabelanger | will check back later once I get some +A action on it | 19:34 |
| pabelanger | :D | 19:34 |
| *** shashank_hegde has quit IRC | 19:36 | |
| *** dimtruck is now known as zz_dimtruck | 19:36 | |
| dgonzalez | Hi all, if somebody has time, i would really appreciate it if you could have a look at https://review.openstack.org/#/c/201316/. (and maybe https://review.openstack.org/#/c/200309/ wich is related to it) | 19:38 |
| *** marekd has quit IRC | 19:40 | |
| *** flashgordon has joined #openstack-infra | 19:41 | |
| *** mdbooth has quit IRC | 19:42 | |
| *** FallenPegasus has quit IRC | 19:42 | |
| *** Sherif has joined #openstack-infra | 19:44 | |
| *** EricGonc_ has joined #openstack-infra | 19:45 | |
| *** ig0r_ has joined #openstack-infra | 19:45 | |
| *** Somay has quit IRC | 19:45 | |
| *** zz_dimtruck is now known as dimtruck | 19:47 | |
| *** FallenPegasus has joined #openstack-infra | 19:47 | |
| *** ig0r_ has quit IRC | 19:47 | |
| *** mrmartin has joined #openstack-infra | 19:48 | |
| *** ddieterl_ has quit IRC | 19:48 | |
| *** EricGonczer_ has quit IRC | 19:49 | |
| *** achanda has quit IRC | 19:52 | |
| *** EricGonc_ has quit IRC | 19:53 | |
| *** Sherif_ has joined #openstack-infra | 19:55 | |
| *** marekd has joined #openstack-infra | 19:55 | |
| *** Sherif has quit IRC | 19:59 | |
| *** dims_ has quit IRC | 20:00 | |
| *** Sherif_ has quit IRC | 20:03 | |
| *** dims_ has joined #openstack-infra | 20:04 | |
| *** SergK has quit IRC | 20:06 | |
| *** ddieterly has joined #openstack-infra | 20:07 | |
| *** mattymo- has quit IRC | 20:08 | |
| *** kozhukalov has quit IRC | 20:08 | |
| *** pasquier-s_ has quit IRC | 20:08 | |
| *** dims_ has quit IRC | 20:09 | |
| *** dimtruck is now known as zz_dimtruck | 20:10 | |
| openstackgerrit | Marton Kiss proposed openstack-infra/puppet-askbot: Add virtualenv support https://review.openstack.org/203378 | 20:10 |
| *** maishsk_ has joined #openstack-infra | 20:10 | |
| *** maishsk has quit IRC | 20:10 | |
| *** maishsk_ is now known as maishsk | 20:10 | |
| *** linkedinyou has quit IRC | 20:11 | |
| *** kozhukalov has joined #openstack-infra | 20:12 | |
| *** pasquier-s has joined #openstack-infra | 20:13 | |
| *** linkedinyou has joined #openstack-infra | 20:15 | |
| *** linkedinyou has quit IRC | 20:16 | |
| *** pasquier-s has quit IRC | 20:18 | |
| *** linkedinyou has joined #openstack-infra | 20:19 | |
| *** kozhukalov has quit IRC | 20:19 | |
| *** linkedinyou has quit IRC | 20:19 | |
| *** _nadya_ has quit IRC | 20:21 | |
| openstackgerrit | Marton Kiss proposed openstack-infra/puppet-askbot: Add virtualenv support https://review.openstack.org/203378 | 20:21 |
| *** _nadya_ has joined #openstack-infra | 20:21 | |
| *** SergK has joined #openstack-infra | 20:22 | |
| *** mrmartin has quit IRC | 20:22 | |
| *** dizquierdo has joined #openstack-infra | 20:22 | |
| lifeless | fungi: can we just install bindep directly, like I started out doing ? | 20:23 |
| *** kozhukalov has joined #openstack-infra | 20:24 | |
| *** mattymo_ has joined #openstack-infra | 20:24 | |
| *** pasquier-s has joined #openstack-infra | 20:25 | |
| *** _nadya_ has quit IRC | 20:26 | |
| openstackgerrit | Marton Kiss proposed openstack-infra/puppet-askbot: Add virtualenv support https://review.openstack.org/203378 | 20:29 |
| *** HeOS has joined #openstack-infra | 20:29 | |
| fungi | lifeless: we can, or we can hold off while the dibification of our devstack-based jobs completes. it's up to you | 20:31 |
| fungi | greghaynes and mordred are supposedly *very* close now | 20:32 |
| greghaynes | fungi: speaking of, I did a check experimental but the results never got submitted to https://review.openstack.org/#/c/198795/ | 20:32 |
| fungi | (close to having that running, i mean. they may be close in other ways too, i have no idea) | 20:32 |
| greghaynes | fungi: im guessing due to restarts of services... any way I can find those logs | 20:32 |
| greghaynes | or I can just wait until the new one I did reports in | 20:33 |
| fungi | greghaynes: you could just check experimental again. i expect jeblair did not restore the experimental pipeline when he restarted zuul | 20:33 |
| lifeless | fungi: I have other changes blocked on that | 20:33 |
| lifeless | fungi: its a usability issue for people submitting req changes | 20:33 |
| greghaynes | ah, ok | 20:33 |
| lifeless | fungi: so if the dibification stuff is done and just percolating through merges, sure. But if it still has work to go (which is my understanding) I'd much rather not delay | 20:34 |
| greghaynes | Yea, thers an unknown amount of work left | 20:34 |
| greghaynes | in theory its not much, but not certain | 20:34 |
| fungi | lifeless: makes sense. if you want to go back to installing bindep within the job while we wait for dib-based workers running devstack jobs in rax, that's okay with me | 20:34 |
| lifeless | whats the review # for the unrevert ? | 20:35 |
| *** achanda has joined #openstack-infra | 20:35 | |
| fungi | apologies again. i didn't realize it was not there yet (i should have checked) | 20:35 |
| fungi | lifeless: 203358 | 20:36 |
| *** mwagner_lap has joined #openstack-infra | 20:36 | |
| fungi | it's just a reproposal of your commit (even still has you as the author) | 20:36 |
| openstackgerrit | lifeless proposed openstack/requirements: Move integration test dependencies into bindep https://review.openstack.org/203358 | 20:37 |
| fungi | transiting between houses again. afk for a bit... | 20:37 |
| lifeless | no apologies needed | 20:37 |
| lifeless | rollbacks to unblock folk are a strategy I encourage | 20:37 |
| lifeless | and forgetting a difference between two deploy strategies is an expected side effect of having two deploy strategies:) | 20:38 |
| *** jsavak has quit IRC | 20:39 | |
| *** mrmartin has joined #openstack-infra | 20:39 | |
| *** esmute has quit IRC | 20:42 | |
| *** thedodd has quit IRC | 20:47 | |
| *** thedodd has joined #openstack-infra | 20:47 | |
| mrmartin | fungi: the venv patch for askbot: https://review.openstack.org/203378 | 20:51 |
| *** puranamr has joined #openstack-infra | 20:58 | |
| *** puranamr has quit IRC | 21:04 | |
| *** amotoki has joined #openstack-infra | 21:05 | |
| *** hdd has joined #openstack-infra | 21:07 | |
| *** puranamr has joined #openstack-infra | 21:08 | |
| *** amotoki has quit IRC | 21:10 | |
| *** esmute has joined #openstack-infra | 21:10 | |
| fungi | mrmartin: saw it. will try to review this evening--thanks for working that out! | 21:12 |
| *** esmute has quit IRC | 21:13 | |
| mrmartin | fungi: it is running well in my vagrant test env, but the review is required. | 21:14 |
| *** hdd has quit IRC | 21:14 | |
| *** dims_ has joined #openstack-infra | 21:15 | |
| greghaynes | looks like the ubuntu-trusty multinode test is going to pass | 21:15 |
| greghaynes | so we might be good to go on trying to switch images again | 21:16 |
| clarkb | what did we have to add to it? | 21:24 |
| greghaynes | we added ping | 21:24 |
| greghaynes | I dont think anything since then | 21:24 |
| *** btully has quit IRC | 21:25 | |
| *** dims_ has quit IRC | 21:26 | |
| fungi | heh... ping? check | 21:27 |
| greghaynes | yes, the minimal part of ubuntu-minimal is not to be taken lightly | 21:27 |
| fungi | maybe devstack (or devstack-gate?) should install ping | 21:27 |
| fungi | in reality it was devstack gate trying to make sure the git server was reachable that was failing on lack of ping | 21:28 |
| fungi | but i don't think devstack-gate currently installs any distro packages and leaves that to devstack (which may not use/expect ping?) | 21:29 |
| greghaynes | No idea, I doubt many have tested running devstack on a host without ping ;) | 21:29 |
| mordred | yay | 21:31 |
| *** puranamr has quit IRC | 21:32 | |
| mordred | (re ubuntu-trusty) | 21:32 |
| *** ddieterly has quit IRC | 21:38 | |
| *** mrmartin has quit IRC | 21:40 | |
| *** FallenPegasus has quit IRC | 21:52 | |
| *** sdake_ has quit IRC | 21:58 | |
| *** dmellado has quit IRC | 22:03 | |
| *** lennyb has quit IRC | 22:04 | |
| *** esmute has joined #openstack-infra | 22:04 | |
| *** dmellado has joined #openstack-infra | 22:05 | |
| *** dizquierdo has quit IRC | 22:14 | |
| *** thedodd has quit IRC | 22:17 | |
| *** harlowja_at_home has joined #openstack-infra | 22:18 | |
| *** esmute has quit IRC | 22:18 | |
| *** yfried_ has joined #openstack-infra | 22:22 | |
| *** flashgordon has quit IRC | 22:23 | |
| *** harlowja_still_a has joined #openstack-infra | 22:24 | |
| *** amotoki has joined #openstack-infra | 22:25 | |
| *** harlowja_at_home has quit IRC | 22:26 | |
| *** harlowja_at_home has joined #openstack-infra | 22:27 | |
| *** harlowja_at_home has quit IRC | 22:28 | |
| *** harlowja_still_a has quit IRC | 22:29 | |
| *** dims_ has joined #openstack-infra | 22:29 | |
| *** amotoki has quit IRC | 22:30 | |
| *** puranamr has joined #openstack-infra | 22:32 | |
| *** dims_ has quit IRC | 22:35 | |
| *** ddieterly has joined #openstack-infra | 22:39 | |
| *** flashgordon has joined #openstack-infra | 22:39 | |
| greghaynes | mordred: argh, ping still not on the images | 22:41 |
| greghaynes | fungi: clarkb https://review.openstack.org/#/c/202116/ | 22:42 |
| *** ddieterly has quit IRC | 22:43 | |
| *** marzif__ has quit IRC | 22:45 | |
| *** sdake has joined #openstack-infra | 22:50 | |
| *** FallenPegasus has joined #openstack-infra | 22:53 | |
| *** jamesmcarthur has joined #openstack-infra | 22:54 | |
| *** marzif_ has joined #openstack-infra | 22:57 | |
| *** Sherif has joined #openstack-infra | 23:00 | |
| *** puranamr has quit IRC | 23:04 | |
| *** puranamr has joined #openstack-infra | 23:04 | |
| *** btully has joined #openstack-infra | 23:05 | |
| *** btully has quit IRC | 23:10 | |
| *** rbrooker has quit IRC | 23:11 | |
| *** ajmiller_ has joined #openstack-infra | 23:12 | |
| *** Sherif has quit IRC | 23:13 | |
| *** ajmiller__ has joined #openstack-infra | 23:16 | |
| *** Sherif has joined #openstack-infra | 23:18 | |
| *** ajmiller_ has quit IRC | 23:19 | |
| *** jamesmcarthur has quit IRC | 23:19 | |
| *** Sherif has quit IRC | 23:19 | |
| *** EricGonczer_ has joined #openstack-infra | 23:20 | |
| *** Sherif has joined #openstack-infra | 23:20 | |
| *** Sherif has quit IRC | 23:21 | |
| openstackgerrit | Merged openstack-infra/system-config: Add ping to our machines https://review.openstack.org/202116 | 23:24 |
| *** amotoki has joined #openstack-infra | 23:26 | |
| *** FallenPegasus has quit IRC | 23:27 | |
| *** FallenPegasus has joined #openstack-infra | 23:28 | |
| *** amotoki has quit IRC | 23:30 | |
| *** FallenPegasus has quit IRC | 23:32 | |
| *** FallenPegasus has joined #openstack-infra | 23:37 | |
| *** jsavak has joined #openstack-infra | 23:39 | |
| *** ddieterly has joined #openstack-infra | 23:39 | |
| *** ddieterly has quit IRC | 23:44 | |
| *** jsavak has quit IRC | 23:44 | |
| openstackgerrit | Merged openstack-infra/puppet-askbot: Add virtualenv support https://review.openstack.org/203378 | 23:56 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!