| *** sdake_ has quit IRC | 00:00 | |
| *** gulic has quit IRC | 00:03 | |
| openstackgerrit | Ian Wienand proposed openstack-infra/nodepool: Convert to use latest statsd version https://review.openstack.org/171890 | 00:04 |
|---|---|---|
| *** hdd has joined #openstack-infra | 00:04 | |
| *** yamada-h has joined #openstack-infra | 00:05 | |
| *** mriedem has joined #openstack-infra | 00:07 | |
| *** yamamoto_ has quit IRC | 00:07 | |
| *** yamada-h has quit IRC | 00:09 | |
| *** sergmelikyan has quit IRC | 00:09 | |
| *** nijaba_ has joined #openstack-infra | 00:10 | |
| *** nijaba has quit IRC | 00:10 | |
| melwitt | I was just reading the scrollback and it looks like I'm seeing the same problem with novaclient that keystonemiddleware is having | 00:11 |
| morganfainberg | melwitt, i think we're going to have this all over the place | 00:11 |
| fungi | melwitt: morganfainberg: also ceilometerclient was running into this today | 00:11 |
| melwitt | gate-tempest-dsvm-neutron-src-python-novaclient-icehouse and gate-tempest-dsvm-neutron-src-python-novaclient-juno are broken because for some reason devstack can't uninstall the new client anymore, I see "Can't uninstall 'python-novaclient'. No files were found to uninstall." error now | 00:12 |
| melwitt | it used to uninstall the latest one and then install an older one that fits global reqs for the branch | 00:12 |
| fungi | yep, those all need corresponding stable branches | 00:12 |
| melwitt | I don't understand what changed | 00:12 |
| *** sputnik13 has quit IRC | 00:13 | |
| clarkb | fungi: I am drinking a breakside ipa now :) | 00:13 |
| fungi | melwitt: well it's a pointless "src" job if it's uninstalling the proposed change you think it's testing so that it can successfully downgrade to an older release | 00:13 |
| fungi | melwitt: instead the library needs a stable branch and needs to not run that src job on changes proposed to the master branch | 00:13 |
| clarkb | its trying to downgrade to test the released version that requirements specifies | 00:14 |
| clarkb | so its not a src job at all without a stable branch | 00:14 |
| melwitt | fungi: true. it's from back when we used to not pin novaclient on stable. however I was thinking this would catch a problem if for some reason someone forgot to cap on stable maybe? | 00:14 |
| fungi | clarkb: i'll enjoy it vicariously through you for now | 00:14 |
| * fungi has a margarita made from patrón anejo, nellie and joe's key lime juice, raw agave nectar and sparkling mineral water | 00:15 | |
| *** annegentle has joined #openstack-infra | 00:16 | |
| clarkb | fungi: the funny thing about that is I think wanderlust ipa is considered by many to be breaksides better ipa | 00:16 |
| fungi | melwitt: that's possible, but probably not worth untangling. we likely just need to press forward cutting stable branches for any libraries capped in stable requirements | 00:16 |
| *** amitgandhinz has quit IRC | 00:17 | |
| clarkb | but I am guessing availability for the non flagship ipa is lower | 00:17 |
| fungi | clarkb: i know, there were several i expected to score a lot higher in the taste test, and several more which scored well that i wouldn't have expected at all | 00:17 |
| fungi | several of the ones i get here turned out to be in the top 15 | 00:18 |
| *** hdd has quit IRC | 00:18 | |
| *** Svedrin has quit IRC | 00:18 | |
| clarkb | fungi: we really should go to breakside during oscon, they have large picnic benches outside (should be nice in july) and their entire range of beer is good and interesting | 00:19 |
| clarkb | fungi: I had a lychee chili saison from them the other day that was interesting | 00:19 |
| fungi | sounds like a great idea | 00:19 |
| fungi | anyway, think it's time for me to pack it in. night all | 00:19 |
| clarkb | good night | 00:19 |
| melwitt | night | 00:20 |
| clarkb | fungi: also no pliny, brave of them | 00:20 |
| *** shashankhegde has joined #openstack-infra | 00:21 | |
| *** mtanino has quit IRC | 00:22 | |
| *** annegentle has quit IRC | 00:22 | |
| *** mahito has joined #openstack-infra | 00:22 | |
| asselin_ | how do you search for statistics on a job, say "gate-tempest-dsvm-neutron-src-python-cinderclient-juno" ? | 00:23 |
| *** Sukhdev has quit IRC | 00:23 | |
| *** Svedrin has joined #openstack-infra | 00:23 | |
| asselin_ | I saw some postings in cinder meeting this morning....http://graphite.openstack.org/ ?? | 00:24 |
| asselin_ | nvm...i think I found it | 00:24 |
| ianw | asselin_: yep, that's the place to look | 00:25 |
| *** chlong has joined #openstack-infra | 00:25 | |
| *** sarob has joined #openstack-infra | 00:28 | |
| *** sarob has joined #openstack-infra | 00:28 | |
| asselin_ | seems that job's been failing since yesterday: http://graphite.openstack.org/render/?width=1074&height=308&_salt=1428539365.551&from=00%3A00_20150401&until=23%3A59_20150408&target=stats_counts.zuul.pipeline.check.job.gate-tempest-dsvm-neutron-src-python-ceilometerclient-juno.SUCCESS&target=stats_counts.zuul.pipeline.check.job.gate-tempest-dsvm-neutron-src-python-ceilometerclient-juno.FAILURE | 00:30 |
| asselin_ | actually this is the one: http://graphite.openstack.org/render/?width=825&height=308&_salt=1428539479.201&from=00%3A00_20150401&until=23%3A59_20150408&target=stats_counts.zuul.pipeline.check.job.gate-tempest-dsvm-neutron-src-python-cinderclient-juno.SUCCESS&target=stats_counts.zuul.pipeline.check.job.gate-tempest-dsvm-neutron-src-python-cinderclient-juno.FAILURE | 00:31 |
| *** marun has quit IRC | 00:43 | |
| openstackgerrit | Clark Boylan proposed openstack-infra/devstack-gate: DO NOT MERGE testing overlay network options https://review.openstack.org/171714 | 00:47 |
| *** bknudson has joined #openstack-infra | 00:48 | |
| clarkb | armax: ^ I raelly couldn't help myself. That fixes a logical condition and I think it will fix mtus which appear ot be the cause of ssh failures on that nova net job | 00:48 |
| greghaynes | clarkb: that mtu | 00:48 |
| *** amotoki has joined #openstack-infra | 00:49 | |
| *** tiswanso has joined #openstack-infra | 00:55 | |
| *** Longgeek has joined #openstack-infra | 00:56 | |
| *** tiswanso has quit IRC | 00:56 | |
| *** annegentle has joined #openstack-infra | 00:57 | |
| *** tiswanso has joined #openstack-infra | 00:57 | |
| *** baoli has quit IRC | 00:57 | |
| armax | clarkb: looking | 00:59 |
| clarkb | armax: I notice that afazekas set the mtu to 1450 on the non dvr job so that value may need tweaking | 00:59 |
| clarkb | armax: but 1476 should accomodate the gre frame | 01:00 |
| clarkb | but its really time to afk now | 01:00 |
| clarkb | dinner soon | 01:00 |
| armax | clarkb: ok, take care | 01:00 |
| *** marcusvrn has joined #openstack-infra | 01:01 | |
| *** ddieterly has joined #openstack-infra | 01:01 | |
| *** rfolco has quit IRC | 01:05 | |
| *** shashankhegde has quit IRC | 01:07 | |
| SpamapS | hrm | 01:10 |
| SpamapS | seems gertty isn't dealing well with os-client-config having been renamed | 01:10 |
| *** hdd has joined #openstack-infra | 01:12 | |
| clarkb | SpamapS: yes it doesn't do that very well. YOu have to manually update the db (or if you are lazy just delete the db and start over) | 01:13 |
| *** crc32 has quit IRC | 01:14 | |
| SpamapS | clarkb: manual seems less lazy since starting over means re-subscribing to things | 01:15 |
| armax | clarkb: whenever you have a chance | 01:15 |
| armax | clarkb: is this change https://review.openstack.org/#/c/171714/6..8/devstack-vm-gate.sh intentional? | 01:15 |
| clarkb | SpamapS: ya but then you have to figure out what the new name is and remember sql update syntax :) | 01:15 |
| armax | clarkb: the one that alters the ovs_gre_bridge command | 01:16 |
| clarkb | armax: yes, the elif is intentional because we were setting up br-ex on nova-net jobs without it | 01:16 |
| clarkb | armax: and the ovs_gre_bridge command was updated because devstack removes the 172.24.4.1/23 and 172.24.4.2/23 addresses anyways | 01:16 |
| clarkb | armax: so I am just not adding those addrs | 01:16 |
| clarkb | armax: if devstack wants us to add them we can flip it back to its old state | 01:16 |
| *** wenlock has joined #openstack-infra | 01:17 | |
| armax | clarkb: okay, I am gonna try the least invasive change to Devstack first | 01:17 |
| armax | clarkb: and see what sticks | 01:17 |
| clarkb | armax: ah ok, in that case feel free to update the ovs_gre_bridge line to be like ps6 | 01:17 |
| clarkb | armax: don't forget to check experimental if you push any patches | 01:17 |
| armax | clarkb: yup | 01:17 |
| *** mtanino has joined #openstack-infra | 01:21 | |
| *** annegentle has quit IRC | 01:22 | |
| *** wenlock has quit IRC | 01:23 | |
| *** tqtran has quit IRC | 01:23 | |
| *** hdd has quit IRC | 01:24 | |
| *** doug-fish has left #openstack-infra | 01:25 | |
| *** mriedem1 has joined #openstack-infra | 01:28 | |
| *** mriedem has quit IRC | 01:28 | |
| *** yamahata has quit IRC | 01:28 | |
| *** asettle is now known as asettle-gymk | 01:29 | |
| *** asettle-gymk is now known as asettle-gym | 01:29 | |
| *** SumitNaiksatam has joined #openstack-infra | 01:31 | |
| armax | clarkb: your PS7 looks pretty good: http://logs.openstack.org/14/171714/7/experimental/check-tempest-dsvm-neutron-aiopcpu-full/46f6059/logs/testr_results.html.gz | 01:33 |
| armax | clarkb: /me wonders why the agent tests barfs | 01:34 |
| *** dboik_ has quit IRC | 01:34 | |
| *** sabari has left #openstack-infra | 01:35 | |
| openstackgerrit | Huang Rui proposed openstack-infra/project-config: Create neutron-zvm-plugin project on StackForge https://review.openstack.org/171030 | 01:38 |
| mordred | SpamapS: ola | 01:45 |
| *** baoli has joined #openstack-infra | 01:45 | |
| *** shashankhegde has joined #openstack-infra | 01:47 | |
| *** otter768 has joined #openstack-infra | 01:49 | |
| *** annegentle has joined #openstack-infra | 01:52 | |
| *** otter768 has quit IRC | 01:53 | |
| *** shashankhegde has quit IRC | 01:53 | |
| *** dims has quit IRC | 01:56 | |
| *** yamada-h has joined #openstack-infra | 01:57 | |
| *** yamada-h has quit IRC | 01:57 | |
| *** mkj_wk has quit IRC | 01:58 | |
| *** baoli has quit IRC | 01:59 | |
| *** wenlock has joined #openstack-infra | 02:02 | |
| *** harlowja is now known as harlowja_away | 02:03 | |
| *** davideagnello has quit IRC | 02:04 | |
| *** smoser has quit IRC | 02:06 | |
| *** mkj_wk has joined #openstack-infra | 02:06 | |
| *** smoser has joined #openstack-infra | 02:06 | |
| *** yamada-h has joined #openstack-infra | 02:09 | |
| *** dangers is now known as dangers_away | 02:11 | |
| *** gulic has joined #openstack-infra | 02:11 | |
| *** baoli has joined #openstack-infra | 02:11 | |
| *** stevemar has joined #openstack-infra | 02:12 | |
| *** amitgandhinz has joined #openstack-infra | 02:13 | |
| timrc | Hey hp server creates are only taking on avg 7 seconds right now :) http://15.125.65.125/dashboard/db/nodepool-performance-hp-cloud | 02:14 |
| *** luqas has joined #openstack-infra | 02:14 | |
| *** mahito_ has joined #openstack-infra | 02:15 | |
| timrc | Oh it heard me and went up to 9 seconds... still the overall avg has dropped into the 20's | 02:15 |
| *** mahito has quit IRC | 02:17 | |
| *** luqas has quit IRC | 02:19 | |
| *** mbacchi-pi has quit IRC | 02:21 | |
| mordred | timrc: openstack cut RCs | 02:23 |
| mordred | timrc: so our overall incoming patch rate is now at the slow period | 02:24 |
| *** annegentle has quit IRC | 02:24 | |
| mordred | which is to say - we are no longer slamming it | 02:24 |
| timrc | mordred, Oh yeah :) I was hoping that was due also to the mass deletes they were apparently going to do. | 02:24 |
| mordred | timrc: ooh! well, let's hope they did those and that that's why | 02:24 |
| mordred | beacuse it woudl be EVERSOGREAT if they just fixed it | 02:24 |
| timrc | mordred, I here they were trying to remove 4 million of 7+ million rows | 02:24 |
| timrc | hear too. | 02:25 |
| mordred | timrc: actually - it looks like the time improvement graph is not tied to the load graph | 02:25 |
| mordred | so - it may be that they did a slow delete over a day or two and it has improved steadily over that time | 02:26 |
| *** yamada-h has quit IRC | 02:30 | |
| *** unicell has quit IRC | 02:30 | |
| openstackgerrit | Merged openstack-infra/shade: Fix volume operations https://review.openstack.org/171319 | 02:31 |
| *** yamada-h has joined #openstack-infra | 02:32 | |
| openstackgerrit | ZhiQiang Fan proposed openstack-dev/hacking: remove h302 from document https://review.openstack.org/171910 | 02:33 |
| *** jogo has quit IRC | 02:34 | |
| *** sabari has joined #openstack-infra | 02:36 | |
| *** yamamoto has joined #openstack-infra | 02:36 | |
| openstackgerrit | Merged openstack-infra/shade: Update os-client-config min version https://review.openstack.org/171337 | 02:40 |
| openstackgerrit | Merged openstack-infra/shade: Fix exception in update_router() https://review.openstack.org/170510 | 02:40 |
| *** mahito has joined #openstack-infra | 02:40 | |
| *** yamahata has joined #openstack-infra | 02:41 | |
| *** mahito_ has quit IRC | 02:43 | |
| *** Longgeek has quit IRC | 02:43 | |
| *** Longgeek has joined #openstack-infra | 02:44 | |
| *** mriedem1 is now known as mriedem_away | 02:48 | |
| *** mahito has quit IRC | 02:54 | |
| *** asettle-gym is now known as asettle | 02:59 | |
| *** ashleighfarnham has joined #openstack-infra | 02:59 | |
| *** amitgandhinz has quit IRC | 03:00 | |
| *** yamada-h has quit IRC | 03:01 | |
| openstackgerrit | Clark Boylan proposed openstack-infra/devstack-gate: DO NOT MERGE testing overlay network options https://review.openstack.org/171714 | 03:03 |
| *** ashleighfarnham has quit IRC | 03:04 | |
| *** yamada-h has joined #openstack-infra | 03:06 | |
| *** alex_xu has quit IRC | 03:08 | |
| *** baoli has quit IRC | 03:09 | |
| *** pal has joined #openstack-infra | 03:11 | |
| *** dhritishikhar_ has joined #openstack-infra | 03:12 | |
| *** baoli has joined #openstack-infra | 03:13 | |
| *** yamada-h has quit IRC | 03:13 | |
| *** xyang1 has quit IRC | 03:13 | |
| *** pal has quit IRC | 03:16 | |
| *** mahito has joined #openstack-infra | 03:19 | |
| armax | clarkb: ping | 03:24 |
| clarkb | pong | 03:24 |
| clarkb | though mostly not here just saw my mtu values were still to big. hopefully new ones stick | 03:24 |
| armax | clarkb: If wonder if setting up the IP’s on the bridges is really necessary | 03:24 |
| clarkb | armax it is for tempest to ssh to VMs | 03:25 |
| clarkb | otherwise packets go to default route which is onto cloud provider network | 03:25 |
| armax | clarkb: if I understand your latest patch correctly though I no longer see br-ex’s being set up | 03:25 |
| clarkb | from there they cannot route to 172.24.5.0/24 | 03:25 |
| clarkb | correct because devstack overrides anyways | 03:26 |
| clarkb | so I did that for simplicity but it wont work completely | 03:26 |
| armax | ok, so the job success is a red-herring? | 03:26 |
| clarkb | kind off, its working because devstack sets 172.24.5.1/24 on controllers br-ex | 03:27 |
| *** gyee has quit IRC | 03:27 | |
| clarkb | whicj is enough I think for packets to go from controller to compute | 03:27 |
| clarkb | but not compute to controller | 03:27 |
| armax | so ultimately the setup should be reinstated, with the proviso that devsatck doesn’t fiddle with it? | 03:27 |
| clarkb | ya | 03:28 |
| armax | which is what we agreed on | 03:28 |
| armax | btw I am chasing why one of the tempest test failures | 03:28 |
| armax | related to L3 agents | 03:29 |
| *** Longgeek_ has joined #openstack-infra | 03:29 | |
| armax | clarkb: what timezone are you? | 03:29 |
| clarkb | pst | 03:29 |
| *** Longgeek has quit IRC | 03:30 | |
| armax | ok, cool, I’ll chase this a bit more before posting the final d-g/ds cleanup | 03:30 |
| *** erlon has quit IRC | 03:31 | |
| clarkb | I think things like shelve and live migration may fail because we need compute to controller routes | 03:32 |
| *** achanda has quit IRC | 03:38 | |
| *** dannywilson has quit IRC | 03:39 | |
| *** marcusvrn has quit IRC | 03:41 | |
| *** ddieterly has quit IRC | 03:41 | |
| *** achanda has joined #openstack-infra | 03:43 | |
| *** yamamoto has quit IRC | 03:44 | |
| *** dims has joined #openstack-infra | 03:45 | |
| *** achanda has quit IRC | 03:45 | |
| tchaypo | lifeless: you probably want to try out https://review.openstack.org/#/c/171457/ | 03:48 |
| tchaypo | my preliminary testing shows that it seems to have removed the blowout problem | 03:48 |
| *** otter768 has joined #openstack-infra | 03:49 | |
| *** jogo has joined #openstack-infra | 03:50 | |
| *** dims has quit IRC | 03:50 | |
| *** davideagnello has joined #openstack-infra | 03:53 | |
| *** otter768 has quit IRC | 03:54 | |
| *** jogo has quit IRC | 03:55 | |
| *** yamahata has quit IRC | 03:55 | |
| lifeless | tchaypo: thanks for the headsup | 03:55 |
| *** yamamoto has joined #openstack-infra | 03:55 | |
| *** yamahata has joined #openstack-infra | 03:55 | |
| *** pal has joined #openstack-infra | 03:57 | |
| *** davideagnello has quit IRC | 03:57 | |
| *** mahito has quit IRC | 03:58 | |
| *** baoli has quit IRC | 03:58 | |
| *** coolsvap|afk is now known as coolsvap | 04:00 | |
| *** sarob has quit IRC | 04:00 | |
| *** tiswanso has quit IRC | 04:01 | |
| *** achanda has joined #openstack-infra | 04:01 | |
| *** mahito has joined #openstack-infra | 04:03 | |
| *** achanda has quit IRC | 04:03 | |
| *** jogo has joined #openstack-infra | 04:05 | |
| *** amitgandhinz has joined #openstack-infra | 04:11 | |
| *** unicell has joined #openstack-infra | 04:13 | |
| *** amitgandhinz has quit IRC | 04:15 | |
| *** luqas has joined #openstack-infra | 04:16 | |
| *** yamamoto has quit IRC | 04:19 | |
| *** luqas has quit IRC | 04:21 | |
| *** btully has joined #openstack-infra | 04:22 | |
| *** achanda has joined #openstack-infra | 04:23 | |
| *** prmtl has joined #openstack-infra | 04:23 | |
| *** koolhead17 has joined #openstack-infra | 04:26 | |
| *** koolhead17 has quit IRC | 04:28 | |
| *** ayoung-mtg is now known as ayoung | 04:30 | |
| *** yamada-h has joined #openstack-infra | 04:31 | |
| *** raginbajin has quit IRC | 04:32 | |
| *** raginbajin has joined #openstack-infra | 04:35 | |
| *** esker has quit IRC | 04:35 | |
| *** sarob has joined #openstack-infra | 04:36 | |
| *** achanda has quit IRC | 04:36 | |
| *** mahito has quit IRC | 04:39 | |
| *** yamamoto has joined #openstack-infra | 04:39 | |
| *** mahito has joined #openstack-infra | 04:40 | |
| *** sarob has quit IRC | 04:41 | |
| *** unicell has quit IRC | 04:42 | |
| *** ddieterly has joined #openstack-infra | 04:42 | |
| *** ddieterly has quit IRC | 04:46 | |
| *** melwitt has quit IRC | 04:47 | |
| *** achanda has joined #openstack-infra | 04:48 | |
| *** dhritishikhar_ has quit IRC | 04:48 | |
| openstackgerrit | Wayne Warren proposed openstack-infra/jenkins-job-builder: Add InvalidAttributeError and MissingAttributeError. https://review.openstack.org/167095 | 04:52 |
| *** stevemar has quit IRC | 04:57 | |
| *** dhritishikhar_ has joined #openstack-infra | 04:57 | |
| *** koolhead17 has joined #openstack-infra | 05:06 | |
| *** mrmartin has joined #openstack-infra | 05:07 | |
| *** prmtl has quit IRC | 05:09 | |
| *** chlong has quit IRC | 05:09 | |
| *** sputnik13 has joined #openstack-infra | 05:10 | |
| flip214 | fungi: thank you very much! | 05:14 |
| *** deepakcs has joined #openstack-infra | 05:15 | |
| flip214 | Good morning, core reviewers: would someone please give https://review.openstack.org/#/c/171754/ a second okay? fungi already gave a +2. | 05:15 |
| *** fedexo has joined #openstack-infra | 05:20 | |
| *** pal has quit IRC | 05:21 | |
| *** yamada-h has quit IRC | 05:21 | |
| *** [HeOS] has quit IRC | 05:22 | |
| *** [HeOS] has joined #openstack-infra | 05:24 | |
| *** crinkle has quit IRC | 05:24 | |
| *** crinkle has joined #openstack-infra | 05:25 | |
| *** koolhead17 has quit IRC | 05:31 | |
| *** mahito has quit IRC | 05:32 | |
| *** mahito has joined #openstack-infra | 05:33 | |
| *** sputnik13 has quit IRC | 05:34 | |
| *** BharatK has joined #openstack-infra | 05:36 | |
| *** ddieterly has joined #openstack-infra | 05:43 | |
| *** ddieterly has quit IRC | 05:49 | |
| *** fedexo has quit IRC | 05:49 | |
| *** otter768 has joined #openstack-infra | 05:50 | |
| openstackgerrit | James Polley proposed openstack-infra/infra-specs: Seperate paragraphs in zuul_split spec https://review.openstack.org/171927 | 05:54 |
| tchaypo | Is cleaning up specs (this was a whitespace-only change) a thing we do? | 05:55 |
| *** otter768 has quit IRC | 05:55 | |
| *** koolhead17 has joined #openstack-infra | 05:55 | |
| *** yamada-h has joined #openstack-infra | 05:57 | |
| AJaeger | flip214, sorry, I prefer some cleanup first... | 05:58 |
| *** jcoufal has joined #openstack-infra | 05:59 | |
| *** spredzy|afk is now known as spredzy | 06:01 | |
| *** ildikov has quit IRC | 06:04 | |
| *** prmtl has joined #openstack-infra | 06:05 | |
| *** mtanino has quit IRC | 06:07 | |
| openstackgerrit | Max Rydahl Andersen proposed openstack-infra/jenkins-job-builder: Add support for 'raw' to inject direct xml to job configs. https://review.openstack.org/160158 | 06:09 |
| *** mahito has quit IRC | 06:10 | |
| openstackgerrit | Marton Kiss proposed openstack-infra/system-config: Fix smartcn analyzer warnings on puppet-run https://review.openstack.org/171600 | 06:15 |
| *** yamahata has quit IRC | 06:16 | |
| *** mahito has joined #openstack-infra | 06:17 | |
| *** luqas has joined #openstack-infra | 06:17 | |
| *** yolanda has quit IRC | 06:18 | |
| *** mrunge has joined #openstack-infra | 06:18 | |
| openstackgerrit | Ph. Marek proposed openstack-infra/project-config: A Devstack-plugin that installs a DRBD backend. https://review.openstack.org/171754 | 06:18 |
| flip214 | AJaeger: ^^ | 06:18 |
| *** mrunge has quit IRC | 06:18 | |
| *** mrunge has joined #openstack-infra | 06:18 | |
| flip214 | please take one more look. | 06:18 |
| *** yolanda has joined #openstack-infra | 06:19 | |
| AJaeger | will do | 06:19 |
| *** yamahata has joined #openstack-infra | 06:20 | |
| AJaeger | flip214, you need to remove the job completely | 06:20 |
| *** mahito has quit IRC | 06:22 | |
| *** btully has quit IRC | 06:22 | |
| openstackgerrit | Marton Kiss proposed openstack-infra/system-config: Fix smartcn analyzer warnings on puppet-run https://review.openstack.org/171600 | 06:22 |
| *** luqas has quit IRC | 06:22 | |
| *** btully has joined #openstack-infra | 06:24 | |
| *** tnovacik has joined #openstack-infra | 06:24 | |
| *** sarob has joined #openstack-infra | 06:25 | |
| *** mahito has joined #openstack-infra | 06:26 | |
| *** sarob has quit IRC | 06:29 | |
| *** sdake_ has joined #openstack-infra | 06:32 | |
| *** prmtl has left #openstack-infra | 06:33 | |
| flip214 | AJaeger: I don't know what I'm doing. Literally. All I'm trying to do is to get an additional plugin for devstack up ... some months ago that would've been a single file in the devstack repository. | 06:35 |
| *** jlibosva has joined #openstack-infra | 06:35 | |
| AJaeger | flip214 just delete those lines that I marked and you should be good | 06:35 |
| AJaeger | I can do it for you as well... | 06:35 |
| flip214 | AJaeger: already on the way. | 06:36 |
| openstackgerrit | Ph. Marek proposed openstack-infra/project-config: A Devstack-plugin that installs a DRBD backend. https://review.openstack.org/171754 | 06:36 |
| flip214 | Yesterday I already complained about the amount of non-code things that have to be done in OpenStack... quite a lot of bureaucracy. | 06:37 |
| AJaeger | flip214, looks fine now. I hear your pain - but wouldn't call it bureaucracy. It's growing pain to handle the scale that OpenStack development is under... | 06:39 |
| *** alex_xu has joined #openstack-infra | 06:40 | |
| flip214 | AJaeger: but adding more and more processes won't help. | 06:42 |
| *** mrmartin has quit IRC | 06:42 | |
| flip214 | because the -core people will have to *help* to fulfill the needs of the new processes. | 06:42 |
| openstackgerrit | Armando Migliaccio proposed openstack-infra/devstack-gate: Configure sub-nodes for Neutron DVR aiopcpu https://review.openstack.org/158525 | 06:42 |
| flip214 | IMO it would be much easier if the cores could autonomously commit... | 06:42 |
| flip214 | so if someone struggles, they should be able to take the parts, fix them, and commit - without needing any other core to acknowledge. | 06:43 |
| flip214 | *that* would save some time. | 06:43 |
| openstackgerrit | Armando Migliaccio proposed openstack-infra/devstack-gate: ***DO NOT MERGE*** Flip DVR bit to collect experimental run datapoints https://review.openstack.org/171809 | 06:44 |
| *** ddieterly has joined #openstack-infra | 06:44 | |
| AJaeger | flip214, the four eyes is part of the OpenStack review process. But different projects handle specific situations different | 06:44 |
| flip214 | AJaeger: it's not 4 eyes, it's 20 eyes sometimes - if different people look at the same patch, reviewing different versions, each one asking for some different change... | 06:45 |
| *** soren_ has joined #openstack-infra | 06:46 | |
| AJaeger | I know that can be frustrating, have seen it myself occasionally. | 06:46 |
| flip214 | thanks for the +2, now I have to wait some hours for fungi, of find some other core... | 06:46 |
| flip214 | "occasionally"? you're lucky. ;) | 06:46 |
| AJaeger | In infra you shouldn't have to wait for that long, we can be pretty quick | 06:47 |
| AJaeger | flip214, I'm hanging out with the friendly guys in openstack-doc and openstack-infra ;) | 06:47 |
| flip214 | ah, different sample. | 06:47 |
| *** alex_xu has quit IRC | 06:47 | |
| *** yamahata has quit IRC | 06:48 | |
| flip214 | no, the people themselves are not the problem. the processes are. | 06:48 |
| *** ddieterly has quit IRC | 06:49 | |
| AJaeger | flip214, people as well: As core reviewer I sometimes give a +2 and tell others: That -1 is invalid IMHO because it's out of scope... | 06:49 |
| *** alex_xu has joined #openstack-infra | 06:51 | |
| openstackgerrit | Wayne Warren proposed openstack-infra/jenkins-job-builder: Add InvalidAttributeError and MissingAttributeError. https://review.openstack.org/167095 | 06:51 |
| flip214 | of course, everybody makes mistakes. But IMO if IBM had such long-drawn processes they would've been dead by 1962. | 06:51 |
| *** trey has quit IRC | 06:53 | |
| *** macjack has joined #openstack-infra | 06:56 | |
| *** Ala has joined #openstack-infra | 06:56 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack-infra/project-config: Normalize projects.yaml https://review.openstack.org/171473 | 06:58 |
| AJaeger | flip214, sorry, don't have time for longer discussion today. These process helped us a lot to get where we are - and there's quite some discussion on eg. the dev mailing list on how to change them in the best way | 06:58 |
| *** trey has joined #openstack-infra | 06:58 | |
| flip214 | yeah, I've seen it. | 07:00 |
| flip214 | I can only hope for the best. | 07:01 |
| flip214 | Thank you very much for your help! | 07:01 |
| AJaeger | flip214, once your project is created, you need to come back here to ask to be added manually to the ACL. If fungi approves, he can do that for you as well... | 07:02 |
| *** MaxV has joined #openstack-infra | 07:02 | |
| AJaeger | flip214, enjoy a sunny day in Austria! | 07:02 |
| *** markus_z has joined #openstack-infra | 07:03 | |
| *** sdake_ has quit IRC | 07:05 | |
| flip214 | AJaeger: TBH, if someone else can do that as well, I'd prefer not to wait 12 hours... | 07:06 |
| flip214 | thanks, you too! | 07:06 |
| AJaeger | flip214, SergeyLukjanov can do it - and fungi is usually up early, so should be 3pm CEST | 07:07 |
| *** sergsh has joined #openstack-infra | 07:10 | |
| *** MaxV has quit IRC | 07:10 | |
| *** e0ne has joined #openstack-infra | 07:11 | |
| flip214 | thanks | 07:12 |
| *** afazekas has joined #openstack-infra | 07:13 | |
| *** armax has quit IRC | 07:13 | |
| *** hichihara has quit IRC | 07:14 | |
| *** trey has quit IRC | 07:14 | |
| *** jistr has joined #openstack-infra | 07:15 | |
| *** doude has joined #openstack-infra | 07:15 | |
| *** mahito has quit IRC | 07:16 | |
| *** dtantsur|afk is now known as dtantsur | 07:17 | |
| *** salv-orlando has joined #openstack-infra | 07:17 | |
| *** sdake has joined #openstack-infra | 07:18 | |
| *** sergsh has quit IRC | 07:20 | |
| *** vlaza has joined #openstack-infra | 07:20 | |
| *** trey has joined #openstack-infra | 07:20 | |
| *** ibiris_away is now known as ibiris | 07:21 | |
| *** ildikov has joined #openstack-infra | 07:22 | |
| *** sergsh has joined #openstack-infra | 07:22 | |
| flip214 | SergeyLukjanov: please give another +2 to https://review.openstack.org/#/c/171754/, thanks a lot | 07:23 |
| *** sdake_ has joined #openstack-infra | 07:25 | |
| *** luqas has joined #openstack-infra | 07:26 | |
| *** sdake has quit IRC | 07:26 | |
| *** salv-orlando has quit IRC | 07:27 | |
| *** salv-orlando has joined #openstack-infra | 07:28 | |
| *** sdake_ has quit IRC | 07:30 | |
| *** salv-orl_ has joined #openstack-infra | 07:30 | |
| *** salv-orlando has quit IRC | 07:31 | |
| *** achanda has quit IRC | 07:31 | |
| GheRivero | morning all | 07:31 |
| *** btully has quit IRC | 07:35 | |
| *** ibiris is now known as ibiris_away | 07:35 | |
| *** wenlock has quit IRC | 07:36 | |
| *** achanda has joined #openstack-infra | 07:37 | |
| *** tnovacik has quit IRC | 07:41 | |
| *** wenlock has joined #openstack-infra | 07:42 | |
| *** ddieterly has joined #openstack-infra | 07:45 | |
| *** achanda has quit IRC | 07:48 | |
| *** ddieterly has quit IRC | 07:49 | |
| *** oomichi has joined #openstack-infra | 07:50 | |
| *** salv-orl_ has quit IRC | 07:50 | |
| *** ibiris_away is now known as ibiris | 07:51 | |
| openstackgerrit | Steven Dake proposed openstack-infra/project-config: Make a check experimental bandit job for Magnum https://review.openstack.org/171954 | 07:51 |
| *** otter768 has joined #openstack-infra | 07:51 | |
| *** e0ne is now known as e0ne_ | 07:52 | |
| *** MaxV has joined #openstack-infra | 07:52 | |
| *** sdake has joined #openstack-infra | 07:52 | |
| *** wenlock has quit IRC | 07:55 | |
| *** otter768 has quit IRC | 07:56 | |
| *** e0ne_ has quit IRC | 07:57 | |
| *** tobiash has quit IRC | 08:00 | |
| *** e0ne has joined #openstack-infra | 08:00 | |
| *** tobiash has joined #openstack-infra | 08:01 | |
| *** jamielennox is now known as jamielennox|away | 08:01 | |
| *** btully has joined #openstack-infra | 08:04 | |
| *** jlanoux has joined #openstack-infra | 08:04 | |
| *** e0ne has quit IRC | 08:06 | |
| *** oomichi has quit IRC | 08:06 | |
| *** btully has quit IRC | 08:09 | |
| *** woodster_ has quit IRC | 08:10 | |
| *** pblaho has joined #openstack-infra | 08:10 | |
| *** derekh has joined #openstack-infra | 08:13 | |
| *** Krinkle|detached is now known as Krinkle | 08:13 | |
| *** arxcruz has joined #openstack-infra | 08:14 | |
| *** tobiash has quit IRC | 08:14 | |
| *** [HeOS] has quit IRC | 08:15 | |
| *** Hal has joined #openstack-infra | 08:15 | |
| *** Hal is now known as Guest71414 | 08:16 | |
| SergeyLukjanov | flip214, AJaeger approved | 08:17 |
| AJaeger | Thanks, SergeyLukjanov. Could you add flip214 to the new ACL as well, please? | 08:17 |
| *** gaelL has quit IRC | 08:17 | |
| SergeyLukjanov | flip214, do you have the same username in gerrit? | 08:18 |
| *** YorikSar has quit IRC | 08:18 | |
| *** tobiash has joined #openstack-infra | 08:18 | |
| flip214 | SergeyLukjanov: no, in gerrit I am philipp.marek@linbit.com | 08:19 |
| AJaeger | SergeyLukjanov, should be phmarek in Gerrit - see http://stackalytics.com/?user_id=philipp-marek | 08:19 |
| flip214 | SergeyLukjanov: thanks a lot | 08:20 |
| flip214 | let's start the next level.... sigh. | 08:20 |
| flip214 | so, in about an hour the repository should be available, right? So then I'll put the template files from http://git.openstack.org/cgit/openstack-dev/devstack-plugin-cookiecutter/tree/ in, fix their content, and try to push again. Is that correct so far? | 08:21 |
| * flip214 just wants to have a few lines in cinder conf during devstack ... | 08:22 | |
| AJaeger | flip214, should be available in 15 mins | 08:23 |
| AJaeger | Yes, that's fine | 08:23 |
| AJaeger | Did you read the infra-manual on this? http://docs.openstack.org/infra/manual/creators.html it gives all details... | 08:23 |
| flip214 | AJaeger: it gives *many* details, not all matching my situation... | 08:24 |
| flip214 | and yes, I'm trying to follow that one. | 08:24 |
| AJaeger | flip214, feel free to continue asking... | 08:24 |
| flip214 | "After the review is approved and groups are created, ask the Infra team to add you to both groups in gerrit, and then you can add other members." | 08:25 |
| AJaeger | flip214, that'S what we just did - asking SergeyLukjanov to do it ;) | 08:25 |
| flip214 | I do not even know which groups this is about... | 08:26 |
| openstackgerrit | Merged openstack-infra/storyboard: Delete all user tokens method https://review.openstack.org/155759 | 08:26 |
| flip214 | "other developers who understand the release process can volunteer to be added as well." -- well, I'm lucky if *I* can find enough time... | 08:26 |
| *** andreykurilin_ has joined #openstack-infra | 08:27 | |
| flip214 | any volunteers? | 08:27 |
| *** mrmartin has joined #openstack-infra | 08:27 | |
| *** mrunge has quit IRC | 08:27 | |
| openstackgerrit | Merged openstack-infra/storyboard: Remove page_size_maximum https://review.openstack.org/167633 | 08:27 |
| openstackgerrit | Merged openstack-infra/project-config: A Devstack-plugin that installs a DRBD backend. https://review.openstack.org/171754 | 08:29 |
| openstackgerrit | Merged openstack-infra/storyboard: Remove page_size_default https://review.openstack.org/167652 | 08:29 |
| *** ildikov has quit IRC | 08:30 | |
| *** jlanoux has quit IRC | 08:30 | |
| sdake | is an experimental job executed with "check experimental" posted in a review? | 08:31 |
| sdake | flip214 which type of project are you making | 08:33 |
| sdake | (out of curiosity) | 08:33 |
| *** jlanoux has joined #openstack-infra | 08:34 | |
| *** zz_johnthetubagu is now known as johnthetubaguy | 08:34 | |
| *** yamamoto has quit IRC | 08:34 | |
| flip214 | sdake: I'm trying to provide a devstack plugin that installs the DRBD backend (for Cinder and Nova) | 08:34 |
| AJaeger | flip214, it'S about the ACLs you created for gerrt - the core and release from https://review.openstack.org/#/c/171754/6/gerrit/acls/stackforge/drbd-devstack.config | 08:34 |
| AJaeger | sdague, yes, experimental should be posted in the review | 08:35 |
| flip214 | AJaeger: thanks, guessed as much already ;) | 08:35 |
| sdake | ajaeger sdake not sdague :) | 08:35 |
| sdake | there are two different cats | 08:35 |
| sdake | btw, thank you for all your thorough reviews and hand holding of my bad project-config hacking :) | 08:35 |
| *** doude has quit IRC | 08:36 | |
| AJaeger | sdake, yes, indeed - sorry about not checking tab completion ;( | 08:36 |
| AJaeger | sdake, you're welcome | 08:36 |
| *** doude has joined #openstack-infra | 08:36 | |
| sdake | oh tab complete second time I've heard that today :) | 08:36 |
| AJaeger | sdake, needs to tabs. sdague only one after pressing "s" :/ | 08:37 |
| AJaeger | I meant two tabs | 08:37 |
| *** heyongli has joined #openstack-infra | 08:38 | |
| *** dizquierdo has joined #openstack-infra | 08:40 | |
| AJaeger | flip214, have fun: http://git.openstack.org/cgit/stackforge/drbd-devstack/ | 08:40 |
| *** cdent has joined #openstack-infra | 08:40 | |
| AJaeger | flip214, wait, not completely setup... | 08:40 |
| *** rwsu has quit IRC | 08:41 | |
| *** jyuso1 has joined #openstack-infra | 08:42 | |
| *** jyuso has joined #openstack-infra | 08:42 | |
| *** YorikSar has joined #openstack-infra | 08:44 | |
| *** teran has quit IRC | 08:44 | |
| *** hashar has joined #openstack-infra | 08:45 | |
| *** ildikov has joined #openstack-infra | 08:45 | |
| *** ddieterly has joined #openstack-infra | 08:46 | |
| *** ssam2 has joined #openstack-infra | 08:47 | |
| *** ddieterly has quit IRC | 08:50 | |
| *** doude has quit IRC | 08:51 | |
| *** doude has joined #openstack-infra | 08:52 | |
| *** jyuso1 has left #openstack-infra | 08:52 | |
| flip214 | AJaeger: I'm starting to pull the template files in. | 08:53 |
| *** gaelL has joined #openstack-infra | 08:55 | |
| *** e0ne has joined #openstack-infra | 08:55 | |
| AJaeger | flip214, cool! | 08:57 |
| *** btully has joined #openstack-infra | 08:58 | |
| *** dtantsur is now known as dtantsur|brb | 08:58 | |
| flip214 | AJaeger: not that I know what I should put in them, but at least I'm kept busy. | 08:59 |
| *** _nadya_ has joined #openstack-infra | 08:59 | |
| flip214 | hmmm, the http://git.openstack.org/cgit/stackforge/devstack-plugin-glusterfs/tree/devstack plugin doesn't have the {pluginname} directory... is that needed or not? | 09:00 |
| *** tobiash has quit IRC | 09:01 | |
| *** btully has quit IRC | 09:02 | |
| *** dhritishikhar_ has quit IRC | 09:04 | |
| flip214 | how would I "install_package" for things not yet upstream? I planned to provide the .deb files along with the plugin and do a "dpkg -i ... && apt-get install -f" for the dependencies ... | 09:06 |
| flip214 | or does install_package know how to deal with local filenames? | 09:06 |
| AJaeger | flip214, check other devstack plugins we have - and I suggest you work with devstack experts. I know even less than you in this area ;( | 09:06 |
| flip214 | thanks anyway. | 09:06 |
| *** koolhead17 has quit IRC | 09:11 | |
| *** SotK has joined #openstack-infra | 09:13 | |
| *** koolhead17 has joined #openstack-infra | 09:15 | |
| *** boris-42 has joined #openstack-infra | 09:16 | |
| *** pcaruana has joined #openstack-infra | 09:17 | |
| *** e0ne is now known as e0ne_ | 09:18 | |
| *** coolsvap is now known as coolsvap|afk | 09:18 | |
| SergeyLukjanov | flip214, I've added you to core and release groups | 09:19 |
| *** yamamoto has joined #openstack-infra | 09:20 | |
| *** claudiub has joined #openstack-infra | 09:23 | |
| flip214 | SergeyLukjanov: thank you very much | 09:24 |
| *** coolsvap|afk is now known as coolsvap | 09:24 | |
| *** e0ne_ is now known as e0ne | 09:25 | |
| *** _shaps_ has joined #openstack-infra | 09:27 | |
| *** ildikov has quit IRC | 09:39 | |
| *** ZZelle has quit IRC | 09:43 | |
| *** ZZelle has joined #openstack-infra | 09:43 | |
| *** ibiris is now known as ibiris_away | 09:45 | |
| *** yamada-h has quit IRC | 09:45 | |
| *** ibiris_away is now known as ibiris | 09:46 | |
| *** sdake_ has joined #openstack-infra | 09:46 | |
| *** ddieterly has joined #openstack-infra | 09:46 | |
| *** sdake_ has quit IRC | 09:49 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config: Cleanup jenkins/jobs/projects.yaml https://review.openstack.org/168148 | 09:49 |
| *** sdake has quit IRC | 09:50 | |
| AJaeger | SergeyLukjanov, could you review https://review.openstack.org/168148 and https://review.openstack.org/168135, please? Those should be trivial cleanups... | 09:50 |
| *** ddieterly has quit IRC | 09:51 | |
| *** koolhead17 has quit IRC | 09:51 | |
| *** btully has joined #openstack-infra | 09:52 | |
| *** dtantsur|brb is now known as dtantsur | 09:52 | |
| *** otter768 has joined #openstack-infra | 09:52 | |
| *** ihrachyshka has joined #openstack-infra | 09:52 | |
| *** _nadya_ has quit IRC | 09:53 | |
| *** ildikov has joined #openstack-infra | 09:54 | |
| *** yamamoto has quit IRC | 09:54 | |
| *** pelix has joined #openstack-infra | 09:55 | |
| *** teran has joined #openstack-infra | 09:56 | |
| *** btully has quit IRC | 09:57 | |
| *** otter768 has quit IRC | 09:57 | |
| *** Krinkle is now known as Krinkle|detached | 10:01 | |
| *** jcoufal has quit IRC | 10:04 | |
| *** coolsvap is now known as coolsvap|afk | 10:06 | |
| *** yamamoto has joined #openstack-infra | 10:14 | |
| *** xnox has quit IRC | 10:15 | |
| *** yamada-h has joined #openstack-infra | 10:16 | |
| openstackgerrit | Jaume Devesa proposed openstack-infra/project-config: Optimise MidoNet devstack-gate job https://review.openstack.org/171994 | 10:16 |
| *** xnox has joined #openstack-infra | 10:17 | |
| openstackgerrit | Joshua Hesketh proposed openstack-infra/zuul: Add support for disabling bad pipelines https://review.openstack.org/154367 | 10:17 |
| *** tosky has joined #openstack-infra | 10:18 | |
| *** Guest71414 has quit IRC | 10:20 | |
| *** isviridov has joined #openstack-infra | 10:20 | |
| *** yamada-h has quit IRC | 10:21 | |
| *** dims_ has joined #openstack-infra | 10:23 | |
| *** wenlock has joined #openstack-infra | 10:24 | |
| tosky | not sure it's managed by infra and if it's a known issue, but ask.openstack.org returns 500 Internal Server Error | 10:27 |
| *** wenlock has quit IRC | 10:29 | |
| *** jcoufal has joined #openstack-infra | 10:29 | |
| *** ibiris is now known as ibiris_away | 10:33 | |
| *** Krinkle|detached is now known as Krinkle | 10:35 | |
| *** jcoufal has quit IRC | 10:36 | |
| *** [HeOS] has joined #openstack-infra | 10:37 | |
| rcarrillocruz | hey folks | 10:37 |
| rcarrillocruz | reading the run_all.sh script from system-config | 10:37 |
| rcarrillocruz | in this line: | 10:37 |
| rcarrillocruz | git reset -q --hard @{u} | 10:37 |
| rcarrillocruz | what the @{u} does? | 10:37 |
| rcarrillocruz | is it some sort of a var expansion? | 10:37 |
| *** tnovacik has joined #openstack-infra | 10:37 | |
| *** jyuso has quit IRC | 10:41 | |
| *** wenlock has joined #openstack-infra | 10:42 | |
| *** jamielennox|away is now known as jamielennox | 10:42 | |
| AJaeger | rcarrillocruz, https://technosorcery.net/blog/2011/12/18/using-gits-at-%7Bupstream%7D-notation/ | 10:44 |
| rcarrillocruz | hah | 10:45 |
| ianw | rcarrillocruz: see gitrevisions manpage, it's shorthand for @{upstream} | 10:45 |
| rcarrillocruz | excellent, my google fu was failing me | 10:45 |
| rcarrillocruz | thx | 10:45 |
| *** btully has joined #openstack-infra | 10:46 | |
| openstackgerrit | Merged openstack-infra/zuul: Update zuul-changes.py doc to match usage https://review.openstack.org/169527 | 10:46 |
| *** deepakcs has quit IRC | 10:47 | |
| *** ddieterly has joined #openstack-infra | 10:47 | |
| *** e0ne is now known as e0ne_ | 10:49 | |
| *** wenlock has quit IRC | 10:50 | |
| *** btully has quit IRC | 10:50 | |
| *** ddieterly has quit IRC | 10:52 | |
| *** _shaps_ has quit IRC | 10:53 | |
| *** _shaps_ has joined #openstack-infra | 10:53 | |
| *** ociuhandu has joined #openstack-infra | 10:57 | |
| openstackgerrit | Joshua Hesketh proposed openstack-infra/puppet-zuul: Move dependencies into lib/ folder https://review.openstack.org/172007 | 10:58 |
| *** redixin has joined #openstack-infra | 10:59 | |
| *** e0ne_ has quit IRC | 10:59 | |
| dims_ | ask.openstack.org is down - dunno if anyone has reported it here yet | 11:03 |
| *** hdd has joined #openstack-infra | 11:04 | |
| *** pc_m has joined #openstack-infra | 11:06 | |
| *** hashar is now known as hasharLunch | 11:06 | |
| *** mrunge has joined #openstack-infra | 11:10 | |
| *** asettle has quit IRC | 11:11 | |
| *** wenlock has joined #openstack-infra | 11:14 | |
| *** gulic has quit IRC | 11:14 | |
| *** e0ne has joined #openstack-infra | 11:15 | |
| *** johnthetubaguy is now known as zz_johnthetubagu | 11:16 | |
| *** amotoki has quit IRC | 11:19 | |
| *** andreykurilin_ has quit IRC | 11:20 | |
| *** ibiris_away is now known as ibiris | 11:21 | |
| *** BharatK has quit IRC | 11:22 | |
| *** BharatK has joined #openstack-infra | 11:22 | |
| *** doude has quit IRC | 11:23 | |
| *** doude has joined #openstack-infra | 11:24 | |
| *** yamamoto has quit IRC | 11:24 | |
| *** BharatK has left #openstack-infra | 11:25 | |
| *** BharatK has quit IRC | 11:25 | |
| *** asettle has joined #openstack-infra | 11:25 | |
| *** wenlock has quit IRC | 11:26 | |
| openstackgerrit | Joshua Hesketh proposed openstack-infra/zuul: Status: Fix JSHint violations and commit settings https://review.openstack.org/169957 | 11:27 |
| *** mrmartin has quit IRC | 11:27 | |
| *** nijaba_ has quit IRC | 11:36 | |
| *** ldnunes has joined #openstack-infra | 11:38 | |
| *** btully has joined #openstack-infra | 11:40 | |
| *** nijaba has joined #openstack-infra | 11:42 | |
| *** nijaba has joined #openstack-infra | 11:42 | |
| *** wenlock has joined #openstack-infra | 11:42 | |
| *** stevemar has joined #openstack-infra | 11:43 | |
| *** BharatK has joined #openstack-infra | 11:43 | |
| *** marcusvrn has joined #openstack-infra | 11:44 | |
| *** btully has quit IRC | 11:44 | |
| *** hogepodge has quit IRC | 11:47 | |
| *** hasharLunch is now known as hashar | 11:48 | |
| *** ddieterly has joined #openstack-infra | 11:48 | |
| *** woodster_ has joined #openstack-infra | 11:49 | |
| *** dkranz has joined #openstack-infra | 11:50 | |
| *** yamamoto has joined #openstack-infra | 11:50 | |
| *** dhritishikhar_ has joined #openstack-infra | 11:50 | |
| *** jamielennox is now known as jamielennox|away | 11:53 | |
| *** ddieterly has quit IRC | 11:53 | |
| *** otter768 has joined #openstack-infra | 11:53 | |
| *** wenlock has quit IRC | 11:54 | |
| *** baoli has joined #openstack-infra | 11:55 | |
| *** baoli has quit IRC | 11:56 | |
| *** baoli has joined #openstack-infra | 11:57 | |
| *** otter768 has quit IRC | 11:58 | |
| *** stevemar has quit IRC | 12:02 | |
| *** hogepodge has joined #openstack-infra | 12:04 | |
| *** yamamoto has quit IRC | 12:06 | |
| *** yamamoto has joined #openstack-infra | 12:07 | |
| *** dprince has joined #openstack-infra | 12:07 | |
| *** dims_ has quit IRC | 12:10 | |
| *** dims_ has joined #openstack-infra | 12:13 | |
| *** liusheng has quit IRC | 12:15 | |
| fungi | dims_: down? checking it now | 12:17 |
| *** mtanino has joined #openstack-infra | 12:18 | |
| *** yamada-h has joined #openstack-infra | 12:19 | |
| *** e0ne is now known as e0ne_ | 12:19 | |
| *** aswadr has joined #openstack-infra | 12:20 | |
| *** bknudson has quit IRC | 12:20 | |
| *** aysyd has joined #openstack-infra | 12:22 | |
| fungi | AJaeger: out of curiosity, why did you object to the -nv version of the drbd job being created? my assumption was that as soon as the non-nv version was passing the -nv version would get added to cinder's check pipeline | 12:22 |
| SergeyLukjanov | AJaeger, +2'd | 12:23 |
| fungi | like with the other devstack plugin projects that also have -nv versions of their integration jobs | 12:23 |
| *** yamada-h has quit IRC | 12:23 | |
| openstackgerrit | Merged openstack-infra/storyboard-webclient: Added bower:install to gruntfile. https://review.openstack.org/171849 | 12:23 |
| *** dims__ has joined #openstack-infra | 12:23 | |
| *** jyuso1 has joined #openstack-infra | 12:24 | |
| *** adalbas has joined #openstack-infra | 12:25 | |
| *** e0ne_ is now known as e0ne | 12:27 | |
| *** dims_ has quit IRC | 12:27 | |
| *** eharney has quit IRC | 12:27 | |
| *** tnovacik has quit IRC | 12:27 | |
| *** asrangne has joined #openstack-infra | 12:27 | |
| *** aswadr has quit IRC | 12:30 | |
| openstackgerrit | yolanda.robla proposed openstack-infra/nodepool: Different queues for slow tasks. https://review.openstack.org/170788 | 12:30 |
| *** hdd has quit IRC | 12:31 | |
| *** asrangne has quit IRC | 12:31 | |
| openstackgerrit | Ian Wienand proposed openstack-infra/nodepool: Convert to use latest statsd version https://review.openstack.org/171890 | 12:32 |
| *** bswartz has quit IRC | 12:32 | |
| *** aswadr has joined #openstack-infra | 12:33 | |
| *** btully has joined #openstack-infra | 12:34 | |
| *** sarob has joined #openstack-infra | 12:34 | |
| openstackgerrit | Merged openstack-infra/project-config: Remove unneccessary noop jobs from zuul/layout.yaml https://review.openstack.org/168135 | 12:35 |
| fungi | okay, so askbot on ask.openstack.org is throwing a traceback in the apache error log with SettingNotSet: ("Startup error, couldn't load MIN_REP.MIN_REP_TO_AUTOAPPROVE_USER", None) | 12:36 |
| *** dhritishikhar_ has quit IRC | 12:37 | |
| fungi | it doesn't look like we upgraded askbot (latest release is from november) | 12:37 |
| *** gordc has joined #openstack-infra | 12:37 | |
| *** btully has quit IRC | 12:39 | |
| *** sarob has quit IRC | 12:39 | |
| dims__ | fungi: we restarted something yesterday? and failed to catch this? | 12:39 |
| fungi | dims__: not that i'm able to find | 12:40 |
| *** Hal has joined #openstack-infra | 12:41 | |
| fungi | trying to figure out what could possibly have changed | 12:41 |
| dims__ | weird | 12:41 |
| *** rfolco has joined #openstack-infra | 12:41 | |
| *** Hal is now known as Guest96211 | 12:41 | |
| *** zz_johnthetubagu is now known as johnthetubaguy | 12:41 | |
| fungi | last commit to the askbot-theme repo was in december | 12:41 |
| fungi | last commit to the puppet-askbot repo was two weeks ago | 12:42 |
| *** bknudson has joined #openstack-infra | 12:42 | |
| *** mriedem_away is now known as mriedem | 12:42 | |
| fungi | last commit to the system-config repo for anything related to askbot was also a couple weeks ago | 12:43 |
| AJaeger | fungi, ah, didn't know about that assumption. For me it was one extra unused job | 12:43 |
| AJaeger | If that assumption would have been documented, I would have approved it ;) | 12:43 |
| fungi | AJaeger: though it's worth noting that if you add a job-group to a project, say the python-jobs group, you'll end up creating all sorts of unused jobs | 12:44 |
| AJaeger | I know... | 12:44 |
| fungi | in this case he was simply copying the job templates instantiated by the other devstack plugins | 12:45 |
| AJaeger | And in that case I prefer to have the job groub instead of the single job | 12:45 |
| AJaeger | fungi, understood | 12:45 |
| fungi | hrm, so it looks like we got quite a few ubuntu package upgrades at 06:50 utc today. seeing if anything is a likely culprit here | 12:46 |
| *** rlandy has joined #openstack-infra | 12:47 | |
| fungi | probably not. the complete list is: libtiff4, libgcrypt11, gpgv, gnupg, facter, libc6-dev, libc-dev-bin, libc-bin, libc6, multiarch-support | 12:47 |
| *** miqui has joined #openstack-infra | 12:48 | |
| *** ddieterly has joined #openstack-infra | 12:49 | |
| yolanda | hi ianw , thx for quick review, still testing some approaches | 12:50 |
| yolanda | so yesterday i had some conversation with jeblair, and we decided to use two queues on a task manager, but if i want that this is not really blocking, we need to parallelize that in some way | 12:51 |
| yolanda | i'm heading for two task managers idea really, i also don't like that approach so much | 12:51 |
| *** jyuso1 has quit IRC | 12:52 | |
| *** ildikov has quit IRC | 12:53 | |
| *** ujuc has joined #openstack-infra | 12:53 | |
| *** ddieterly has quit IRC | 12:53 | |
| *** aswadr has quit IRC | 12:54 | |
| fungi | aha! i think i've found it | 12:55 |
| *** weshay has joined #openstack-infra | 12:55 | |
| *** _nadya_ has joined #openstack-infra | 12:55 | |
| *** _nadya_ has quit IRC | 12:55 | |
| *** aswadr has joined #openstack-infra | 12:55 | |
| *** tnovacik has joined #openstack-infra | 12:55 | |
| fungi | django-redis-cache got a new release uploaded to pypi today | 12:55 |
| *** _nadya_ has joined #openstack-infra | 12:56 | |
| fungi | ahh, nope, we didn't upgrade to it though | 12:56 |
| *** sarob has joined #openstack-infra | 12:58 | |
| *** mtanino has quit IRC | 13:01 | |
| fungi | also redis seems to be complaining that memory>maxmemory | 13:01 |
| *** Krinkle is now known as Krinkle|detached | 13:02 | |
| *** bswartz has joined #openstack-infra | 13:03 | |
| *** e0ne is now known as e0ne_ | 13:04 | |
| *** eharney has joined #openstack-infra | 13:06 | |
| *** lifeless has quit IRC | 13:07 | |
| *** spzala has joined #openstack-infra | 13:08 | |
| *** ildikov has joined #openstack-infra | 13:08 | |
| *** adalbas has quit IRC | 13:09 | |
| *** tiswanso has joined #openstack-infra | 13:09 | |
| *** yamahata has joined #openstack-infra | 13:10 | |
| *** rkukura has quit IRC | 13:10 | |
| *** macjack1 has joined #openstack-infra | 13:12 | |
| *** macjack1 has quit IRC | 13:12 | |
| *** mwagner_lap has quit IRC | 13:12 | |
| fungi | okay, so ask.openstack.org is back up now. i had to change the maxmemory in /etc/redis/redis.conf from 256m to 512m | 13:16 |
| *** zaneb has joined #openstack-infra | 13:17 | |
| fungi | unfortunately i'm not sure why that's necessary, given that the setting on the old ask server was only 128m | 13:17 |
| fungi | as soon as mrmartin is around i'll pick his brain about it | 13:17 |
| zaneb | sdague: could I trouble you to reapprove https://review.openstack.org/#/c/165433/ (tripleo-common repo creation) now that the TC has approved the new name? | 13:19 |
| fungi | zaneb: done | 13:20 |
| sdague | zaneb: done | 13:20 |
| openstackgerrit | Li Ma proposed openstack/requirements: Support the latest release of oslo.messaging https://review.openstack.org/172045 | 13:20 |
| zaneb | fungi, sdague: much appreciated, thank you guys! | 13:20 |
| *** _nadya_ has quit IRC | 13:22 | |
| *** e0ne_ is now known as e0ne | 13:25 | |
| *** dhritishikhar_ has joined #openstack-infra | 13:25 | |
| *** amitgandhinz has joined #openstack-infra | 13:26 | |
| *** dims__ has quit IRC | 13:28 | |
| *** dustins has joined #openstack-infra | 13:28 | |
| *** ociuhandu_ has joined #openstack-infra | 13:28 | |
| *** dims_ has joined #openstack-infra | 13:28 | |
| *** btully has joined #openstack-infra | 13:28 | |
| *** ociuhandu has quit IRC | 13:30 | |
| *** ociuhandu_ is now known as ociuhandu | 13:30 | |
| *** soren_ has quit IRC | 13:31 | |
| *** btully has quit IRC | 13:33 | |
| *** doude has quit IRC | 13:33 | |
| *** ddieterly has joined #openstack-infra | 13:34 | |
| *** django__ has joined #openstack-infra | 13:35 | |
| *** doude has joined #openstack-infra | 13:35 | |
| *** chillisantiago has joined #openstack-infra | 13:36 | |
| *** dims__ has joined #openstack-infra | 13:36 | |
| openstackgerrit | Thanh Ha proposed openstack-infra/jenkins-job-builder: Add support for Sonatype CLM plugin https://review.openstack.org/164446 | 13:37 |
| *** dims_ has quit IRC | 13:37 | |
| *** tnovacik has quit IRC | 13:37 | |
| *** dboik has joined #openstack-infra | 13:40 | |
| openstackgerrit | Merged openstack-infra/project-config: Create tripleo-common project under TripleO https://review.openstack.org/165433 | 13:43 |
| *** dougshelley66 has joined #openstack-infra | 13:45 | |
| *** Krinkle|detached is now known as Krinkle | 13:45 | |
| *** stevemar has joined #openstack-infra | 13:46 | |
| *** sabeen has joined #openstack-infra | 13:46 | |
| *** dhritishikhar_ has quit IRC | 13:47 | |
| *** dhritishikhar_ has joined #openstack-infra | 13:47 | |
| fungi | ttx: when you get a moment, can you tag openstack/governance master branch tip as april-2015-elections? | 13:47 |
| ttx | fungi: sure, doing it now while I remember | 13:48 |
| ttx | aka GTD 2-min rule | 13:48 |
| *** jcoufal has joined #openstack-infra | 13:48 | |
| fungi | heh | 13:49 |
| ttx | fungi: should be ok, please doublecheck | 13:50 |
| fungi | ttx: lgtm, thanks! | 13:50 |
| *** ujuc has quit IRC | 13:52 | |
| *** dhritishikhar___ has joined #openstack-infra | 13:53 | |
| *** ujuc has joined #openstack-infra | 13:53 | |
| *** openstackgerrit has quit IRC | 13:53 | |
| *** openstackgerrit has joined #openstack-infra | 13:53 | |
| *** sc` has joined #openstack-infra | 13:54 | |
| *** otter768 has joined #openstack-infra | 13:54 | |
| *** dhritishikhar_ has quit IRC | 13:55 | |
| *** ddieterly has quit IRC | 13:55 | |
| *** ddieterly has joined #openstack-infra | 13:56 | |
| *** sarob has quit IRC | 13:56 | |
| *** mwagner_lap has joined #openstack-infra | 13:58 | |
| *** markvoelker has quit IRC | 13:58 | |
| *** otter768 has quit IRC | 13:59 | |
| *** markvoelker has joined #openstack-infra | 13:59 | |
| *** fhubik has joined #openstack-infra | 14:03 | |
| *** jerryz has joined #openstack-infra | 14:04 | |
| fhubik | Hey, could someone help me with review.openstack.org issue? My nick appears in review for multiple times for some reason. PM & thx ;) | 14:05 |
| fungi | fhubik: you're going to have to be more specific. can you provide an example? | 14:06 |
| *** yamamoto has quit IRC | 14:08 | |
| fhubik | fungi: Yes I can, in this (https://review.openstack.org/#/c/31782/24) review there is my name and email two times, that is probably not ok | 14:09 |
| *** mtanino has joined #openstack-infra | 14:09 | |
| *** sdake has joined #openstack-infra | 14:10 | |
| tosky | it happened to me when there was some trouble in the launchpad login; it seems it's not possible to add me as reviewer, maybe similar issue | 14:10 |
| *** erlon has joined #openstack-infra | 14:10 | |
| fungi | fhubik: can you elaborate on what on that page is your name and email? where on that page do you see it "two times" that you think it's in error? | 14:11 |
| *** kgiusti has joined #openstack-infra | 14:11 | |
| *** koolhead17 has joined #openstack-infra | 14:13 | |
| sc` | i see the same name twice in that review | 14:13 |
| *** wenlock has joined #openstack-infra | 14:13 | |
| fhubik | fungi: Therein section "Reviewer" is my exact name and email one time, and after one line same name and same email again | 14:13 |
| sc` | close to the bottom of the reviewer list | 14:13 |
| fungi | fhubik: ahh, i see it. yes it looks like you're Filip Hubík. i think you have two gerrit accounts | 14:13 |
| *** heyongli has quit IRC | 14:14 | |
| fungi | fhubik: you didn't say what your name was, so i was looking for a needle in a haystack there | 14:14 |
| *** heyongli has joined #openstack-infra | 14:15 | |
| fhubik | fungi: Yeah, sorry ;) Yes, even that's possible - I have credentials only for single account | 14:15 |
| fungi | fhubik: you have account id 14345 created 2014-12-15 16:05:30 and account id 14687 created 2015-01-16 11:44:04 | 14:16 |
| fhubik | fungi: Can it be merged somehow? | 14:16 |
| fungi | fhubik: no, but i can deactivate one of them. did you maybe change the e-mail address you were logging into launchpad with back in january? it looks like there are two different launchpad openids, one for each of these accounts | 14:17 |
| *** pcaruana has quit IRC | 14:17 | |
| *** _nadya_ has joined #openstack-infra | 14:17 | |
| *** lifeless has joined #openstack-infra | 14:18 | |
| *** radez_g0n3 is now known as radez | 14:18 | |
| fungi | the older account seems to have used your gmail address to log into lp, and the new account uses your redhat e-mail address | 14:18 |
| fungi | the old account has your redhat address as an additional e-mail, so i'm guessing the newer account is the one we should deactivate | 14:19 |
| *** sabeen3 has joined #openstack-infra | 14:19 | |
| openstackgerrit | Cyril Roelandt proposed openstack-infra/project-config: python-saharaclient: make the py3 gate voting https://review.openstack.org/172076 | 14:19 |
| fhubik | fungi: Yes I had to do some ping-pong with accounts, so if it would be possible, then you can deactivate the older "gmail" one | 14:19 |
| fungi | fhubik: when you go to https://review.openstack.org/#/settings/ do you see 14345 or 14687 as your account id there? | 14:20 |
| *** sabeen has quit IRC | 14:20 | |
| fhubik | fungi: I see 14687 | 14:21 |
| *** esker has joined #openstack-infra | 14:21 | |
| fungi | fhubik: okay, so you're currently logging in as the newer account. i need to determine if you have any existing changes we'll be orphaning since chances are those are still being uploaded with the username/key tied to your old account | 14:22 |
| fhubik | fungi: Thanks! | 14:22 |
| *** radez is now known as radez_bbl | 14:23 | |
| *** wenlock has quit IRC | 14:24 | |
| fungi | fhubik: yeah, there's no username, ssh keys, contact info, cla, et cetera for that newer 14687 account you're using now. i'll see if i can redirect the openid for it to 14345 in the database and then disable the newer one | 14:24 |
| *** AJaeger has quit IRC | 14:26 | |
| *** jcoufal has quit IRC | 14:26 | |
| fhubik | fungi: ok | 14:26 |
| *** fhubik has quit IRC | 14:27 | |
| *** jcoufal has joined #openstack-infra | 14:27 | |
| *** MarkAtwood has joined #openstack-infra | 14:28 | |
| *** zz_dimtruck is now known as dimtruck | 14:28 | |
| dkranz | Can some one explain why this has not gone into the gate queue? It seems to have the necessary reviews, approvals, etc. with no dependencies: https://review.openstack.org/#/c/166955/ | 14:29 |
| *** jcoufal has quit IRC | 14:29 | |
| *** afazekas has quit IRC | 14:29 | |
| *** markus_z has quit IRC | 14:30 | |
| *** achanda has joined #openstack-infra | 14:32 | |
| *** wenlock has joined #openstack-infra | 14:33 | |
| *** wenlock_ has joined #openstack-infra | 14:34 | |
| fungi | dkranz: it looks like at the time it was approved, its parent change hadn't been approved yet. though i'm not yet sure why it didn't get pulled into the gate when that happened | 14:34 |
| *** e0ne is now known as e0ne_ | 14:34 | |
| *** e0ne_ is now known as e0ne | 14:35 | |
| openstackgerrit | Julia Kreger proposed openstack-infra/shade: Change Ironic node lookups to support names https://review.openstack.org/171164 | 14:35 |
| *** mriedem has quit IRC | 14:37 | |
| *** doug-fish has joined #openstack-infra | 14:39 | |
| *** btully has joined #openstack-infra | 14:39 | |
| *** bhunter71 has joined #openstack-infra | 14:40 | |
| *** dhritishikhar_ has joined #openstack-infra | 14:40 | |
| *** johnchalekson has joined #openstack-infra | 14:40 | |
| *** mattfarina has joined #openstack-infra | 14:40 | |
| *** achanda has quit IRC | 14:41 | |
| *** salv-orlando has joined #openstack-infra | 14:41 | |
| *** asselin has joined #openstack-infra | 14:43 | |
| *** tnovacik has joined #openstack-infra | 14:44 | |
| *** dhritishikhar___ has quit IRC | 14:44 | |
| *** ildikov is now known as ildikov_afk | 14:46 | |
| *** isviridov has quit IRC | 14:46 | |
| openstackgerrit | Brant Knudson proposed openstack/requirements: Uncap pymongo https://review.openstack.org/172095 | 14:46 |
| *** yamamoto has joined #openstack-infra | 14:47 | |
| *** harlowja_at_home has joined #openstack-infra | 14:47 | |
| fungi | dkranz: no luck. zuul debug logs from the time 171246 was approved say that 166955 can't be enqueued into the gate but don't mention why it believed that http://paste.openstack.org/show/201328/ | 14:47 |
| *** jerryz has quit IRC | 14:48 | |
| openstackgerrit | Brant Knudson proposed openstack/requirements: Uncap pymongo https://review.openstack.org/172095 | 14:48 |
| fungi | dkranz: anyway, a second approval on that change (or getting mtreinish to remove and readd his) should get 166955 right into the gate at this point | 14:49 |
| Adri2000 | https://ask.openstack.org/ 500; known issue? | 14:49 |
| mtreinish | fungi: looks like sdague already added a +A on it to take care of that | 14:49 |
| fungi | Adri2000: it happened earlier today and i doubled the redis memory limit to get it back online. i'll take another look but that's not good if redis is rapidly ballooning | 14:50 |
| bknudson | I posted a change to uncap pymongo... not sure what the best thing to do here is. | 14:52 |
| *** erikmwil_ has joined #openstack-infra | 14:52 | |
| *** erikmwilson is now known as Guest81531 | 14:52 | |
| *** erikmwil_ is now known as erikmwilson | 14:52 | |
| *** erikmwilson_ has joined #openstack-infra | 14:52 | |
| fungi | Adri2000: thanks for the heads up. i've redoubled the max memory for it. looks like we must have that file under configuration management | 14:53 |
| *** salv-orlando has quit IRC | 14:54 | |
| *** jcoufal has joined #openstack-infra | 14:54 | |
| *** emagana has joined #openstack-infra | 14:55 | |
| *** amotoki has joined #openstack-infra | 14:57 | |
| openstackgerrit | Jeremy Stanley proposed openstack-infra/system-config: Increase redis maxmemory to 512m on ask.o.o https://review.openstack.org/172105 | 14:58 |
| *** devvesa has joined #openstack-infra | 14:58 | |
| fungi | that ^ should take care of it, and i've set it by hand and disabled puppet agent on the server in the meantime | 14:58 |
| *** jlibosva1 has joined #openstack-infra | 15:00 | |
| *** baoli has quit IRC | 15:00 | |
| *** jlibosva1 has quit IRC | 15:00 | |
| *** jlibosva1 has joined #openstack-infra | 15:00 | |
| *** dboik_ has joined #openstack-infra | 15:00 | |
| *** isviridov has joined #openstack-infra | 15:00 | |
| *** johnchalekson has quit IRC | 15:00 | |
| *** mrmartin has joined #openstack-infra | 15:01 | |
| *** baoli has joined #openstack-infra | 15:01 | |
| *** marun has joined #openstack-infra | 15:02 | |
| *** isviridov has quit IRC | 15:02 | |
| *** jlibosva has quit IRC | 15:02 | |
| *** achanda has joined #openstack-infra | 15:02 | |
| *** dboik has quit IRC | 15:04 | |
| *** jcoufal has quit IRC | 15:06 | |
| *** sergsh has quit IRC | 15:06 | |
| *** ddieterly has quit IRC | 15:07 | |
| *** tonytan4ever has joined #openstack-infra | 15:07 | |
| *** isviridov has joined #openstack-infra | 15:08 | |
| *** dangers_away is now known as dangers | 15:09 | |
| *** isviridov has quit IRC | 15:09 | |
| *** Kiall has quit IRC | 15:10 | |
| *** yamahata has quit IRC | 15:10 | |
| *** isviridov has joined #openstack-infra | 15:10 | |
| *** achanda has quit IRC | 15:11 | |
| *** Kiall has joined #openstack-infra | 15:12 | |
| anteaya | sigmavirus24_awa: are you at pycon? | 15:12 |
| derekh | I'd like to switch the tripleo fedora jobs to F21, image building is the first step if anybody is around to take a look https://review.openstack.org/#/c/169778/ | 15:12 |
| *** wenlock__ has joined #openstack-infra | 15:13 | |
| *** ddieterly has joined #openstack-infra | 15:15 | |
| *** jistr is now known as jistr|mtg | 15:15 | |
| *** ddieterl_ has joined #openstack-infra | 15:15 | |
| sileht | sdague, hi, I have an issue with the new pip_install_gr: http://logs.openstack.org/62/172062/1/check/gate-oslo.messaging-dsvm-functional-amqp1-f21/5c5048a/logs/devstacklog.txt.gz | 15:15 |
| sileht | sdague, the pyngus packages is not part of global requirements because it can't be installed on ubuntu vms | 15:16 |
| sdague | sileht: can you link the actual line? | 15:16 |
| sileht | sdague, sure: http://logs.openstack.org/62/172062/1/check/gate-oslo.messaging-dsvm-functional-amqp1-f21/5c5048a/logs/devstacklog.txt.gz#_2015-04-09_14_12_17_297 | 15:16 |
| *** sarob has joined #openstack-infra | 15:16 | |
| sdague | sileht: so, devstack in tree really needs to filter these packages through global requirements because of version pin breakage | 15:17 |
| *** dannywilson has joined #openstack-infra | 15:17 | |
| sdague | why can't pyngus be installed on ubuntu? | 15:17 |
| sileht | sdague, missing libs .h | 15:18 |
| *** cdent has quit IRC | 15:18 | |
| *** shashankhegde has joined #openstack-infra | 15:18 | |
| sileht | (python-qpid-proton) | 15:18 |
| *** cdent has joined #openstack-infra | 15:18 | |
| *** ddieterly has quit IRC | 15:19 | |
| sdague | ok, I'm still not sure why that means it can't be in gr | 15:19 |
| *** vlaza has left #openstack-infra | 15:19 | |
| *** dannywil_ has joined #openstack-infra | 15:19 | |
| ttx | mordred: probably a common question, but missed the answer so far... how is Infra affected long term by the HP public cloud announcement ? | 15:19 |
| *** dannywilson has quit IRC | 15:19 | |
| ttx | jeblair: ^ | 15:19 |
| sdague | here is the problem, now that we are capping things in gr for stable, devstack needs to be able to install the gr version otherwise it's all going to explode | 15:19 |
| *** dhritishikhar_ has quit IRC | 15:20 | |
| sileht | sdague, should I submit a change to global-requirement, backport it to kilo with pinning ? | 15:20 |
| *** dhritishikhar_ has joined #openstack-infra | 15:20 | |
| morganfainberg | ttx, ping | 15:20 |
| sdague | sileht: that seems reasonable to me, ttx, what do you think? | 15:21 |
| morganfainberg | ttx, wanted to ask a question re: stable branches (juno and icehouse) for the *-clients/middlware, etc | 15:21 |
| ttx | sdague: given what I just posted on the stable library thread, I guess that makes sense | 15:21 |
| *** mattt has joined #openstack-infra | 15:21 | |
| morganfainberg | ttx, because right now due to gate jobs, we can't land things in the master(s) for keystoneclient/middleware/novaclient. this is because the requirements can't be resolved between master and the juno gate job. | 15:22 |
| ttx | morganfainberg: I'm in my office doing incubated projects syncs | 15:22 |
| ttx | I'll brb | 15:22 |
| morganfainberg | ttx, no worries. | 15:22 |
| *** armax has joined #openstack-infra | 15:22 | |
| sileht | sdague, ttx thanks, I will do that, and test it with 'DependsOn:' to see if that some my issue | 15:23 |
| sdague | sileht: sounds good | 15:23 |
| *** AJaeger has joined #openstack-infra | 15:23 | |
| mordred | ttx: there is no HP Cloud announcement | 15:24 |
| mordred | ttx: that's a poorly written article based around an out of context quote followed up by a poorly written PR response | 15:24 |
| cody-somerville | ttx: "Note that the "master" branch of Heat is now open for Liberty development, and feature freeze restrictions no longer apply there !" <-- this was kind of unexpected and confusing to me. | 15:25 |
| sdague | sileht: I'm actually kind of confused why you are using devstack to just install proton | 15:26 |
| mordred | cody-somerville: once we cut the first RC for a project, RC development goes into a release-specific branch which means master can re-open | 15:26 |
| sdague | it seems like it would be a lot simpler to just have a script that does that | 15:26 |
| sdague | as you are using no other parts of devstack | 15:26 |
| *** jistr|mtg is now known as jistr | 15:27 | |
| sdague | sileht: yeh, I'm actually completely confused why you are using devstack here :) | 15:27 |
| cody-somerville | It makes it pretty confusing if you manually have to select a bunch of different branches to build from. | 15:27 |
| mtreinish | clarkb: there is a parallel neutron pg job somewhere, it might be neutron-pg-full | 15:28 |
| mordred | cody-somerville: can you explain why that's something you'd need to do? | 15:28 |
| mtreinish | clarkb: that job wasn't parallel because it didn't work before and they didn't want to lose pg coverage | 15:28 |
| mordred | cody-somerville: (just trying to understand the pain) | 15:28 |
| *** johnchalekson has joined #openstack-infra | 15:28 | |
| fungi | cody-somerville: if your intent is tracking kilo, then prefer branches in this order: stable/kilo, proposed/kilo, master | 15:28 |
| mordred | fungi: ++ | 15:29 |
| *** tnovacik has quit IRC | 15:29 | |
| *** ildikov_afk has quit IRC | 15:29 | |
| cody-somerville | I want to build a distro of OpenStack with all the kilo bits. | 15:29 |
| mordred | cody-somerville: yah. - do what fungi just said | 15:29 |
| fungi | cody-somerville: once a proposed/kilo branch appears for a project, master starts moving on toward liberty | 15:29 |
| *** yamamoto has quit IRC | 15:29 | |
| openstackgerrit | Sean M. Collins proposed openstack-infra/devstack-gate: Force openvswitch as the Neutron agent for Ironic https://review.openstack.org/172118 | 15:29 |
| *** Ala has quit IRC | 15:29 | |
| ttx | cody-somerville: you should track proposed/kilo | 15:29 |
| cody-somerville | Yea, that was unexpected to happen on a per-project level instead of everyone forking to say proposed/kilo together. | 15:30 |
| sdague | cody-somerville: it always happens in this rolling mechanism | 15:30 |
| mordred | cody-somerville: it should not be unexpected - it's been this way since forever | 15:30 |
| *** wenlock__ has quit IRC | 15:30 | |
| fungi | it happens mostly together, within a few days | 15:30 |
| cody-somerville | or it happening this early. | 15:30 |
| ttx | If only everybody was raedy at the same time :) | 15:30 |
| *** asselin has quit IRC | 15:30 | |
| mordred | cody-somerville: but luckily the logic that fungi describes is very simple | 15:30 |
| *** e0ne is now known as e0ne_ | 15:30 | |
| ttx | It's pretty much how it's been since the dawn of time though | 15:30 |
| *** dboik_ has quit IRC | 15:30 | |
| cody-somerville | It doesn't quite work since we want to continue to consume liberty as well. | 15:30 |
| mordred | ttx: we did change the name of the proposed branch a little wile ago | 15:31 |
| mordred | cody-somerville: you just said nonsense | 15:31 |
| ttx | right, for clarity | 15:31 |
| ttx | mordred: +1 | 15:31 |
| *** dboik has joined #openstack-infra | 15:31 | |
| mordred | cody-somerville: if you want to consume kil | 15:31 |
| cody-somerville | We were hoping there to be a cut off point where we'd create our own stable/kilo to continue kilo | 15:31 |
| mordred | kilo | 15:31 |
| mordred | cody-somerville: there is | 15:31 |
| mordred | cody-somerville: listen to teh logic | 15:31 |
| ttx | cody-somerville: that might be a good reading: https://wiki.openstack.org/wiki/ReleaseCycle | 15:31 |
| mordred | cody-somerville: if you want kilo | 15:31 |
| cody-somerville | but we're not ready!! :P | 15:31 |
| mordred | you should look for stable/kilo, if it exists, use it, if it does not but proposed/kilo exists, use it, if it does not use master | 15:31 |
| mordred | if you want liberty, do the same logic except s/kilo/liberty/ | 15:32 |
| mordred | if your syncing script can't handle that | 15:32 |
| david-lyle | how can I uncouple the python26 jobs from django_openstack_auth in project-config? it seems to be treated like the other libraries and it's not entirely clear to me how to uncouple python26 from running | 15:32 |
| mordred | fix the syncing script | 15:32 |
| mordred | because the logic is very simple and will ALWAYS work | 15:32 |
| ttx | mordred: ah? You're not stopping the HPCloud just yet ? | 15:32 |
| mordred | ttx: no | 15:32 |
| *** spredzy is now known as spredzy|afk | 15:32 | |
| ttx | mordred: good to know. | 15:32 |
| mordred | ttx: it was a hachet piece | 15:32 |
| ttx | If only we had journalists... oh wait | 15:32 |
| openstackgerrit | Sean M. Collins proposed openstack-infra/devstack-gate: Force openvswitch as the Neutron agent for Ironic https://review.openstack.org/172118 | 15:33 |
| david-lyle | the non-compete announcement was made months ago, not sure why the fires are being restoked | 15:33 |
| mordred | ttx: which along with the BI mckenty-loving "openstack must be dead because VCs haven't made a billion" make for a great week for PR | 15:33 |
| sdague | ttx: you mean if only journalists actually fact checked | 15:33 |
| fungi | david-lyle: we can remove python 2.6 jobs from that project if it no longer needs to support python 2.6. is django_openstack_auth not used by horizon in icehouse? | 15:33 |
| mordred | david-lyle: I'm guessing that people wanted to jump on the flame-bait bandwagon that BI started | 15:33 |
| ttx | sdague: no fact-checking means no journalism to me | 15:33 |
| *** harlowja_at_home has quit IRC | 15:34 | |
| sdague | ttx: facts are so 20th century | 15:34 |
| cody-somerville | mordred: the problem is actually CI test logic + logic to create branches automatically when we detect them upstream. | 15:34 |
| ttx | sdague: never let facts get in the way of clickbait | 15:34 |
| david-lyle | fungi: in icehouse yes, but the version is capped, same with juno | 15:34 |
| mordred | cody-somerville: ok ... but why can't you add a simple if/else/else instead of a hardcoded branch name? | 15:34 |
| fungi | david-lyle: so we need to still run python 2.6 jobs on any backport changes that are proposed to the stable/icehouse branch of django_openstack_auth i guess? | 15:34 |
| fungi | cody-somerville: fwiw, devstack-gate has this logic built into it already | 15:35 |
| david-lyle | fungi, yes that would be optimal, but to this point we haven't backported to d-o-a | 15:35 |
| cody-somerville | Yea, we broke that I'm pretty sure with our own contrived mess. | 15:35 |
| mordred | cody-somerville: for branch in stable/$release proposed/$release master; do if $(git branch -a | grep upstream/$branch) ; then return $branch ; fi | 15:36 |
| fungi | david-lyle: we could limit that job so that it only runs for stable/icehouse branch changes on django_openstack_auth, since the only way you're going to get a fix around the requirements cap is going to be to add a stable branch for it | 15:36 |
| mordred | cody-somerville: I'm really not kidding ^^ | 15:36 |
| *** e0ne_ is now known as e0ne | 15:36 | |
| pkarikh | AJaeger: hello! Could you please take a look at updated patchset here? https://review.openstack.org/#/c/171117/ Your comment has been addressed. | 15:36 |
| *** salv-orlando has joined #openstack-infra | 15:36 | |
| mordred | cody-somerville: make that a function anywhere you currently have logic that says "pull upstream kilo" and you'll be fine | 15:36 |
| mordred | cody-somerville: the mess you make with the internal branching based on that shouldn't matter | 15:37 |
| openstackgerrit | David Lyle proposed openstack-infra/project-config: Removing py26 from django_openstack_auth in Kilo https://review.openstack.org/172123 | 15:37 |
| david-lyle | fungi: that was my attempt, but I don't think the job I'm conditionally running is named correctly | 15:37 |
| AJaeger | pkarikh: done, +2 | 15:38 |
| pkarikh | AJaeger: thank you! | 15:38 |
| fungi | david-lyle: oh, perhaps. i'll take a look in a bit, though my dance card is already pretty full for today | 15:39 |
| david-lyle | fungi, sure, just the job names seem to be built programmatically, but didn't want to just run with that assumption | 15:39 |
| *** jistr has quit IRC | 15:40 | |
| *** ihrachyshka has quit IRC | 15:41 | |
| fungi | david-lyle: we already have a lot of whitelisting in place because we stopped running gate-{name}-python26 jobs on changes for the api servers except on stable/icehouse. i just need to see whether we need to add django_openstack_auth to a whitelist for that or subtract it from a blacklist for that | 15:41 |
| *** koolhead17 has quit IRC | 15:41 | |
| ctlaugh | I have a change (https://review.openstack.org/#/c/171300/) that originally had a +1 from XenServer CI, but now has a -1 and no comment explaining why / logs / etc. Any ideas? | 15:42 |
| *** Swami has joined #openstack-infra | 15:44 | |
| *** dimtruck is now known as zz_dimtruck | 15:44 | |
| fungi | ctlaugh: click the "toggle ci" button at the bottom of the page to reveal the ci comments | 15:45 |
| fungi | ctlaugh: looks like it has a link to logs | 15:45 |
| ctlaugh | fungi: Ah, thank you | 15:45 |
| *** mriedem has joined #openstack-infra | 15:47 | |
| fungi | ctlaugh: according to http://dd6b71949550285df7dc-dda4e480e005aaa13ec303551d2d8155.r49.cf1.rackcdn.com/00/171300/2/14869/run_tests.log it failed tempest.scenario.test_snapshot_pattern.TestSnapshotPattern.test_snapshot_pattern[compute,id-608e604b-1d63-4a82-8e3e-91bc665c90b4,image,network] | 15:47 |
| ctlaugh | fungi: I'm going to trigger a recheck -- I think there error has to be transient, or something unrelated to this patch | 15:48 |
| fungi | ctlaugh: entirely possible | 15:49 |
| *** adalbas has joined #openstack-infra | 15:49 | |
| fungi | ctlaugh: also no idea whether the devstack reviewers pay close attention to the Citrix XenServer CI account | 15:49 |
| fungi | BobBall_AWAY: (when he's not _AWAY) may be able to help you figure out the problem if it's persistent | 15:50 |
| *** tkelsey has joined #openstack-infra | 15:54 | |
| *** otter768 has joined #openstack-infra | 15:55 | |
| *** sputnik13 has joined #openstack-infra | 15:56 | |
| *** dtantsur is now known as dtantsur|afk | 15:57 | |
| sdague | fungi: I don't, it's random fail so often, and it's not reporting in the standardized format so it's not in the CI results table | 15:58 |
| *** MaxV has quit IRC | 15:58 | |
| *** dustins has quit IRC | 15:58 | |
| openstackgerrit | Victor Stinner proposed openstack/requirements: Bump eventlet version to 0.17.3 https://review.openstack.org/172132 | 15:58 |
| *** gyee has joined #openstack-infra | 15:59 | |
| *** otter768 has quit IRC | 16:00 | |
| *** hashar has quit IRC | 16:01 | |
| *** rwsu has joined #openstack-infra | 16:02 | |
| zaro | morning | 16:02 |
| *** yamahata has joined #openstack-infra | 16:02 | |
| *** zz_dimtruck is now known as dimtruck | 16:03 | |
| *** radez_bbl is now known as radez | 16:05 | |
| *** jlanoux has quit IRC | 16:06 | |
| *** vdo has joined #openstack-infra | 16:08 | |
| *** dizquierdo has quit IRC | 16:11 | |
| *** derekh has quit IRC | 16:12 | |
| *** isviridov has quit IRC | 16:12 | |
| openstackgerrit | Merged openstack/requirements: Cap pymongo < 3.0 https://review.openstack.org/171438 | 16:12 |
| *** isviridov has joined #openstack-infra | 16:13 | |
| *** isviridov has quit IRC | 16:14 | |
| *** wenlock__ has joined #openstack-infra | 16:14 | |
| *** jlibosva1 has quit IRC | 16:15 | |
| *** shashankhegde has quit IRC | 16:15 | |
| *** esker has quit IRC | 16:16 | |
| pleia2 | good morning | 16:16 |
| *** dkehn has joined #openstack-infra | 16:16 | |
| *** dhritishikhar_ has quit IRC | 16:16 | |
| *** devvesa has quit IRC | 16:16 | |
| *** signed8bit has joined #openstack-infra | 16:18 | |
| *** redixin has quit IRC | 16:19 | |
| *** luqas has quit IRC | 16:24 | |
| *** wenlock__ has quit IRC | 16:24 | |
| *** vdo has quit IRC | 16:25 | |
| *** melwitt has joined #openstack-infra | 16:26 | |
| cinerama | pleia2: are we ready to server today? | 16:26 |
| pleia2 | cinerama: at the very least, we need this one merged https://review.openstack.org/#/c/169074/ | 16:27 |
| pleia2 | otherwise we have nothing to deploy :) | 16:27 |
| pleia2 | would be nice to have this one in too https://review.openstack.org/#/c/171790/ | 16:27 |
| cinerama | pleia2: heh that is true | 16:27 |
| cinerama | well, *i* think those are ready enough | 16:28 |
| * pleia2 nods | 16:28 | |
| pleia2 | cinerama: can you update the link in https://review.openstack.org/#/c/171791/ ? | 16:28 |
| pleia2 | then I can +2 that one | 16:28 |
| cinerama | pleia2: oh well spotted, let me just test that real quick | 16:29 |
| pleia2 | if we can get other cores to +A, I can launch the server this afternoon | 16:30 |
| pleia2 | I have calls at 10 and 1, and a dr appt at 11:15, but by 1:30 pacific I should be free! | 16:30 |
| *** radez is now known as radez_g0n3 | 16:30 | |
| * pleia2 mutters about thursday | 16:30 | |
| *** sarob has quit IRC | 16:30 | |
| fungi | pleia2: tristanC: i just finished preparing the electoral rolls for nova, glance and tc elections and sent them to you | 16:31 |
| pleia2 | fungi: thanks \o/ | 16:31 |
| tristanC | fungi: that's very appreciated, thanks! | 16:31 |
| *** Krinkle is now known as Krinkle|detached | 16:33 | |
| mtreinish | pcrews: I was wondering if you had any more thoughts on the best way to do the data expiration for the subunit2sql db | 16:33 |
| *** ashleighfarnham has joined #openstack-infra | 16:33 | |
| mtreinish | we're getting close to the release so it's probably time to start thinking about that | 16:34 |
| *** dguitarbite has quit IRC | 16:34 | |
| *** cdent has quit IRC | 16:34 | |
| openstackgerrit | Stephanie Miller proposed openstack-infra/puppet-zanata: Upgrade wildfly puppet module https://review.openstack.org/171791 | 16:34 |
| *** harlowja_away is now known as harlowja | 16:35 | |
| *** ayoung is now known as ayoung-lunch | 16:36 | |
| *** signed8b_ has joined #openstack-infra | 16:37 | |
| cinerama | pleia2: so i had a chat with one of the zanata folx yesterday - we may need to whip up our own login page to cover using the "internal" accounts with openid | 16:38 |
| pcrews | mtreinish: probably your idea of the cron job to wipe out records > 6 months old seems the simplest / soundest idea for that :) We did something similar for the lbaas-database, just a config with like max_record_age and a job to wipe out appropriate records daily. | 16:39 |
| pleia2 | cinerama: ah, good to know | 16:39 |
| openstackgerrit | Khai Do proposed openstack-infra/gerritlib: Update project docs https://review.openstack.org/165979 | 16:39 |
| *** signed8b_ has quit IRC | 16:40 | |
| *** signed8bit has quit IRC | 16:40 | |
| mtreinish | pcrews: ok, wfm. That should be easy to do, we probably could add an option to do the data purge into one of the commands too. | 16:41 |
| *** signed8bit has joined #openstack-infra | 16:41 | |
| pcrews | mtreinish: ++ I like that idea (put it into a command) / would be useful. | 16:41 |
| *** patrickeast has joined #openstack-infra | 16:42 | |
| *** tonytan4ever has quit IRC | 16:42 | |
| *** signed8bit has quit IRC | 16:43 | |
| cinerama | pleia2: oh by the way, how do we handle upgrading puppet modules in general? i note we're on a rather old version of the mysql one but it might be pretty painful to upgrade..there are some big changes there | 16:43 |
| *** signed8bit has joined #openstack-infra | 16:43 | |
| *** dannywil_ has quit IRC | 16:43 | |
| *** Krinkle|detached is now known as Krinkle | 16:45 | |
| *** signed8bit has quit IRC | 16:45 | |
| *** cdent has joined #openstack-infra | 16:45 | |
| *** signed8bit has joined #openstack-infra | 16:46 | |
| openstackgerrit | Sean M. Collins proposed openstack-infra/devstack-gate: Force openvswitch as the Neutron agent for Ironic https://review.openstack.org/172118 | 16:49 |
| *** signed8bit has quit IRC | 16:50 | |
| *** signed8bit has joined #openstack-infra | 16:50 | |
| *** dannywilson has joined #openstack-infra | 16:51 | |
| *** signed8bit has quit IRC | 16:52 | |
| *** signed8bit has joined #openstack-infra | 16:52 | |
| *** davideagnello has joined #openstack-infra | 16:53 | |
| *** unicell has joined #openstack-infra | 16:53 | |
| pleia2 | cinerama: the mysql one is hard (part why it's so outdated), but typically one of us reads the changelogs and does testing to make sure the new module doesn't break anything | 16:53 |
| zaro | fungi: would you like to help me upgrade review-dev to 2.10? | 16:53 |
| pleia2 | I don't remember specifically if the mysql one has real blockers | 16:54 |
| *** _nadya_ has quit IRC | 16:54 | |
| *** signed8bit has quit IRC | 16:54 | |
| *** unicell has left #openstack-infra | 16:54 | |
| *** signed8bit has joined #openstack-infra | 16:54 | |
| SpamapS | pleia2: I thought we used RAX Trove for our databases. ? | 16:55 |
| cinerama | i think if we switch to a newer one we'll have to update things that use it | 16:55 |
| pleia2 | SpamapS: I don't think we use it for everything | 16:55 |
| pleia2 | cinerama: I think there's a fair amount of "if it ain't broke... I have better things to do" | 16:56 |
| clarkb | morning | 16:56 |
| pleia2 | morning clarkb | 16:56 |
| *** aswadr has quit IRC | 16:57 | |
| cinerama | pleia2: good thing i'm too stupid to know better. :) | 16:57 |
| pleia2 | pfft | 16:57 |
| *** shashankhegde has joined #openstack-infra | 16:58 | |
| *** e0ne is now known as e0ne_ | 16:59 | |
| *** sushilkm has joined #openstack-infra | 17:00 | |
| fungi | zaro: i'll see what i can do after i get out of this meeting that just started | 17:00 |
| sushilkm | hello .... | 17:00 |
| openstackgerrit | Dmitry Ratushnyy proposed openstack-infra/git-review: git review -sv gets more verbose https://review.openstack.org/172147 | 17:00 |
| sushilkm | i needed an account to setup a 3rd party CI for Trove, can someone help | 17:01 |
| *** Krinkle is now known as Krinkle|detached | 17:01 | |
| zaro | sushilkm: have you read http://ci.openstack.org/third_party.html ? | 17:01 |
| krtaylor | sushilkm, are you just getting started? | 17:01 |
| SpamapS | pleia2: "If it aint broke" == "It's not automated so I am unwilling to disturb the bear's slumber." | 17:02 |
| sushilkm | i m trying to setup a 3rd party ci for a new datastore in trove ... and found that only a CI account could post a verified status | 17:03 |
| *** e0ne_ is now known as e0ne | 17:03 | |
| *** tiswanso has quit IRC | 17:03 | |
| *** signed8bit has quit IRC | 17:03 | |
| *** signed8bit has joined #openstack-infra | 17:03 | |
| krtaylor | sushilkm, the link zaro pointed to is a good place to start, read that, then talk to the trove team, then get involved in the meetings | 17:04 |
| zaro | fungi: np. we just need to get these two changes approved: https://review.openstack.org/#/c/171876 & https://review.openstack.org/#/c/171732 preferablly in that order. should be just rubber stamp. | 17:04 |
| fungi | k | 17:04 |
| *** ssam2 has quit IRC | 17:04 | |
| *** smccully has joined #openstack-infra | 17:05 | |
| armax | clarkb: ping | 17:05 |
| clarkb | armax: pong | 17:05 |
| krtaylor | sushilkm, also anteaya runs a great office hours meeting on Mondays and Tuesdays, its a good place to discuss initial troubles -> https://wiki.openstack.org/wiki/Meetings/ThirdParty | 17:05 |
| krtaylor | sushilkm, but start with the documentation, it will answer a lot of questions | 17:06 |
| armax | clarkb: hey I tried what we discussed yesterday and the test outcome is not promising http://logs.openstack.org/09/171809/2/experimental/check-tempest-dsvm-neutron-aiopcpu-full/02dc41e/ | 17:06 |
| armax | clarkb: I’ve been building a repro env locally to better see what’s going on | 17:06 |
| *** e0ne has quit IRC | 17:06 | |
| clarkb | armax: I think its mtus | 17:06 |
| armax | clarkb: maskeraded as ssh failures? | 17:07 |
| clarkb | armax: nova net is suffering the same problems as well | 17:07 |
| clarkb | armax: yes | 17:07 |
| clarkb | armax: aiui the old linux bridge gre tunnel configures mtus properly automagically for us and did fragmentation | 17:07 |
| clarkb | but ovs wants to rely on pmtud except there is no source for an icmp packet to be sent from across an l2 link | 17:08 |
| clarkb | so I feel a little better about my dislike of ovs right now | 17:08 |
| armax | clarkb: :) | 17:08 |
| *** signed8bit has quit IRC | 17:13 | |
| clarkb | armax: I want to say the simplest fix is to update the cirros version in devstack and tell dhcp to configure an mtu of ~1450 in the cirros VMs | 17:13 |
| fungi | zaro: presumably the job update will need to propagate and then we can recheck the gerrit change | 17:13 |
| *** Krinkle|detached is now known as Krinkle | 17:13 | |
| *** signed8bit has joined #openstack-infra | 17:13 | |
| openstackgerrit | Akihiro Motoki proposed openstack/requirements: Update python-neutronclient to 2.4.0 https://review.openstack.org/172149 | 17:13 |
| *** tiswanso has joined #openstack-infra | 17:13 | |
| clarkb | armax: but I am also trying to get an env up to poke around and confirm my suspiciions | 17:13 |
| clarkb | I love when software projects decide to do the "correct" thing at the expense of doing the "working" thing | 17:13 |
| sushilkm | ok thanks | 17:13 |
| armax | clarkb: incidentally Neutron added the ability to advertise MTU via dhcp in Kilo, but I guess that won’t help Nova | 17:13 |
| *** chillisantiago has quit IRC | 17:13 | |
| openstackgerrit | Swaminathan Vasudevan proposed openstack-infra/devstack-gate: Configure sub-nodes for Neutron DVR aiopcpu https://review.openstack.org/158525 | 17:13 |
| zaro | fungi: makes sense. ohh and i forgot i will need to push another change to actually update gerrit with the newly generated 2.10 build but i guess i can't until after those changes go in. | 17:13 |
| *** openstack has quit IRC | 17:13 | |
| *** openstack has joined #openstack-infra | 17:15 | |
| -sendak.freenode.net- [freenode-info] channel trolls and no channel staff around to help? please check with freenode support: http://freenode.net/faq.shtml#gettinghelp | 17:15 | |
| Swami | clarkb: are you talking about the addition of the "multi_node" flag. | 17:15 |
| armax | clarkb: agreed…I thought I’d give it a try nonetheless | 17:15 |
| *** signed8bit has quit IRC | 17:15 | |
| *** signed8bit has joined #openstack-infra | 17:16 | |
| *** ihrachyshka has joined #openstack-infra | 17:16 | |
| clarkb | Swami: yes, MULTI_HOST and PUBLIC_INTERFACE are both noops there | 17:17 |
| *** achanda has joined #openstack-infra | 17:17 | |
| clarkb | Swami: only nova network makes use of that in devstack | 17:17 |
| Swami | clarkb: ok I will go ahead and remove it. | 17:17 |
| *** radez_g0n3 is now known as radez | 17:17 | |
| Swami | clarkb: I was with the impression that is used for neutron as well. Good to know. | 17:17 |
| *** yamamoto has joined #openstack-infra | 17:18 | |
| openstackgerrit | Merged openstack-infra/project-config: Run Gerrit unit tests https://review.openstack.org/171876 | 17:18 |
| armax | clarkb: actually PUBLIC_INTERFACE is used in neutron | 17:18 |
| *** ujuc has quit IRC | 17:18 | |
| armax | it’s the MULTI_HOST flag that’s nova only | 17:18 |
| *** pelix has quit IRC | 17:18 | |
| Swami | armax: Yes that is true. But by default it will be using "br-ex". | 17:19 |
| Swami | If we don't change the "br-ex" name then we should be good, I think. | 17:19 |
| *** baoli has quit IRC | 17:19 | |
| wenlock_ | hey, hoping i can get some help with jjb template question. If i have info I need my jjb macro to set, like a password or token, is there a way to describe that setting in a jjb template and have jjb store it in jenkins management view as a value to use on a config script? | 17:20 |
| armax | smccully: the default is not br-ex | 17:20 |
| *** doude has quit IRC | 17:20 | |
| clarkb | armax: its not use here | 17:21 |
| clarkb | armax: it is only used when doing provider networks | 17:21 |
| *** amotoki has quit IRC | 17:21 | |
| armax | clarkb: true | 17:21 |
| *** doude has joined #openstack-infra | 17:21 | |
| Swami | armax:clarkb: will remove the change. | 17:21 |
| *** spredzy|afk is now known as spredzy | 17:22 | |
| clarkb | armax: I think it would be nice if lib/neutron (not legacy) added that interface to the br-ex bridge for us | 17:23 |
| *** yamamoto has quit IRC | 17:23 | |
| *** dustins has joined #openstack-infra | 17:24 | |
| *** luqas has joined #openstack-infra | 17:24 | |
| openstackgerrit | Swaminathan Vasudevan proposed openstack-infra/devstack-gate: Configure sub-nodes for Neutron DVR aiopcpu https://review.openstack.org/158525 | 17:24 |
| *** salv-orlando has quit IRC | 17:24 | |
| clarkb | armax: on my old test nodes from yesterday I set the mtu to 1450 on br_pub (for nova net) and am able to ssh from controller to vm on compute, compute to vm on controller, and vm on controller to vm on compute | 17:26 |
| clarkb | armax: so not sure why my mtu changes aren't happy when set up by d-g, will have to see if those changes are sticking | 17:26 |
| armax | clarkb: indeed | 17:27 |
| smccully | armax do what now? | 17:28 |
| *** boris-42 has quit IRC | 17:28 | |
| armax | smccully: um…sorry, I meant to ping Swami | 17:28 |
| smccully | no worries | 17:28 |
| Swami | armax: hi | 17:28 |
| *** baoli has joined #openstack-infra | 17:29 | |
| *** luqas has quit IRC | 17:29 | |
| *** SumitNaiksatam has quit IRC | 17:29 | |
| clarkb | wenlock_: I don't understand your question | 17:30 |
| clarkb | wenlock_: can you expand a bit on "have jjb store it in jenkins management view as a value to use on a config script" | 17:31 |
| wenlock_ | clarkb in my case, im trying to push an app to something like helion dev platform env, that requires a password. | 17:31 |
| wenlock_ | clarkb, so i have a jjb macro where i have a bash script to do that | 17:31 |
| *** dboik has quit IRC | 17:31 | |
| wenlock_ | clarkb we then have a puppet erb template, pulling in the password from eyaml, but jjb when it creates the job, will store that password on the job in the clear, as it's suppose to. | 17:32 |
| wenlock_ | clarkb i want to store that password somewhere better | 17:32 |
| wenlock_ | clarkb, so it's not in the clear | 17:32 |
| clarkb | wenlock_: the way we do that is by having puppet write the secret data on disk of the slave node | 17:33 |
| clarkb | wenlock_: jenkins is not really a good credentials manager | 17:33 |
| armax | clarkb: so is there a chance we can get hold of the nodes running on https://jenkins04.openstack.org/job/check-tempest-dsvm-neutron-aiopcpu-full/23/ | 17:33 |
| *** YorikSar has quit IRC | 17:33 | |
| armax | clarkb: they should be helpful to see what’s going on | 17:33 |
| clarkb | armax: sure | 17:34 |
| armax | clarkb: on my local env the changes we make in d-g seem to stick | 17:34 |
| armax | clarkb: after the devstack run I meam | 17:34 |
| *** tonytan4ever has joined #openstack-infra | 17:35 | |
| *** tonytan4ever has quit IRC | 17:36 | |
| *** emagana has quit IRC | 17:37 | |
| *** cdent has quit IRC | 17:38 | |
| *** emagana has joined #openstack-infra | 17:38 | |
| *** spredzy is now known as spredzy|afk | 17:38 | |
| wenlock_ | clarkb, so even though jenkins has a credentials option to store that kind of stuff, im guessing jjb doesn't manage that... still better idea to get that from a secret folder (in practice anyway)...right? | 17:39 |
| *** rkukura has joined #openstack-infra | 17:40 | |
| *** gyee has quit IRC | 17:40 | |
| clarkb | wenlock_: jenkins credentials are not currently manageable by jjb. | 17:41 |
| clarkb | it doesn't have to be a secret folder | 17:41 |
| clarkb | it just needs to be unnaccessible by things that shouldn't access it | 17:41 |
| *** tonytan4ever has joined #openstack-infra | 17:41 | |
| clarkb | (note jenkins is a bad secret manager because slaves apparently can tel lthe master to do things and the master happily goes along with it, I would not trust the master to keep secrets from arbitrary slaves) | 17:41 |
| clarkb | this is one of our motivations for partitioning our jenkins masters | 17:42 |
| wenlock_ | clarkb, cool ... thats helpful insight | 17:42 |
| *** emagana has quit IRC | 17:43 | |
| wenlock_ | clarkb ty | 17:43 |
| openstackgerrit | yolanda.robla proposed openstack-infra/nodepool: Different queues for slow tasks. https://review.openstack.org/170788 | 17:45 |
| *** dboik has joined #openstack-infra | 17:46 | |
| fungi | wenlock_: in fact, we have a separate jenkins master for workers which have access to account secrets, and the jobs we run on those don't ever execute untrusted code | 17:46 |
| fungi | wenlock_: specifically because of that | 17:46 |
| clarkb | armax: the frustrating thing here is I think we are very very close :) hopefully we don't have to update teh cirros image and set dhcp options (I don't expect that to be simple) | 17:46 |
| fungi | apparently jenkins development finally acknowledged that this was a "bad thing[tm]" and introduced features a few months ago to limit slave access to masters | 17:47 |
| *** rmcall has joined #openstack-infra | 17:47 | |
| clarkb | fungi: oh thats good news, didn't know about that | 17:47 |
| *** signed8bit has quit IRC | 17:47 | |
| fungi | but it's been a long time coming, and is new enough i wouldn't trust it yet | 17:47 |
| pleia2 | if anyone wants to browse https://review.openstack.org/#/q/status:open+topic:zanata,n,z it would be helpful (aiming for launching the -dev server this afternoon, need some of these | 17:47 |
| armax | clarkb: I am trying to prove your theory but all of the sudden my tunnel got busted and I am redoing some of the setup | 17:47 |
| * pleia2 off to dr appt | 17:47 | |
| clarkb | pleia2: I will take a look while I wait on devstack and tempest | 17:48 |
| pleia2 | clarkb: thanks! | 17:48 |
| *** ivar-lazzaro has joined #openstack-infra | 17:48 | |
| *** salv-orlando has joined #openstack-infra | 17:51 | |
| *** SumitNaiksatam has joined #openstack-infra | 17:51 | |
| *** Sukhdev has joined #openstack-infra | 17:52 | |
| openstackgerrit | yolanda.robla proposed openstack-infra/nodepool: Different queues for slow tasks. https://review.openstack.org/170788 | 17:52 |
| *** e0ne has joined #openstack-infra | 17:53 | |
| yolanda | jeblair, mordred, initial approach for the 2-queue feature in nodepool, what are your thoughts? | 17:53 |
| *** e0ne is now known as e0ne_ | 17:53 | |
| openstackgerrit | Stephanie Miller proposed openstack-infra/puppet-zanata: Upgrade wildfly puppet module https://review.openstack.org/171791 | 17:55 |
| *** ildikov_afk has joined #openstack-infra | 17:55 | |
| *** otter768 has joined #openstack-infra | 17:56 | |
| *** e0ne_ is now known as e0ne | 17:57 | |
| *** baoli has quit IRC | 17:59 | |
| *** baoli has joined #openstack-infra | 17:59 | |
| openstackgerrit | Russell Bryant proposed openstack-infra/reviewstats: Drop myself from nova-core https://review.openstack.org/172160 | 18:00 |
| *** ZZelle_ has joined #openstack-infra | 18:00 | |
| *** dizquierdo has joined #openstack-infra | 18:01 | |
| *** teran has quit IRC | 18:01 | |
| *** otter768 has quit IRC | 18:01 | |
| *** spredzy|afk is now known as spredzy | 18:01 | |
| *** baoli has quit IRC | 18:01 | |
| *** [HeOS] has quit IRC | 18:01 | |
| clarkb | cinerama: pleia2 comments on https://review.openstack.org/#/c/169074/ not the most urgent but I think it helps clarify how things should go in | 18:02 |
| openstackgerrit | Merged openstack-infra/system-config: Upgrade wildfly puppet module https://review.openstack.org/171790 | 18:03 |
| *** e0ne is now known as e0ne_ | 18:03 | |
| openstackgerrit | David Caro proposed openstack-infra/jenkins-job-builder: Added parallelization options https://review.openstack.org/75514 | 18:04 |
| mrmartin | fungi: hi | 18:04 |
| clarkb | cinerama: oh I think I needed to let 171790 merge before 171791 would pass tests? | 18:04 |
| fungi | mrmartin: hi there. so you're in favor of 172105 looks like? | 18:05 |
| *** doude has quit IRC | 18:05 | |
| fungi | mrmartin: i'll see if i can drum up some quick approval for it in that case | 18:05 |
| *** patrickeast_ has joined #openstack-infra | 18:05 | |
| mrmartin | fungi if that solves the problem | 18:05 |
| mrmartin | have you seen something in the logs? | 18:05 |
| *** sdake_ has joined #openstack-infra | 18:05 | |
| *** doude has joined #openstack-infra | 18:06 | |
| *** patrickeast has quit IRC | 18:06 | |
| *** patrickeast_ is now known as patrickeast | 18:06 | |
| mrmartin | what is the memory size of that instance? | 18:06 |
| fungi | mrmartin: any guesses as to why redis started overrunning its 256mb memory limit today? i notice the old ask server was 128mb | 18:06 |
| *** sarob has joined #openstack-infra | 18:06 | |
| mrmartin | fungi: I don't have any idea | 18:06 |
| mrmartin | askbot using that as a memory cache | 18:06 |
| fungi | luckily i sized the new server for 4gb ram instead of the old server's 2gb so we can increase this safely | 18:06 |
| clarkb | the first rule of redis is that it will run out of memory | 18:07 |
| mrmartin | let's go with the new setting, let's see what happens | 18:07 |
| fungi | mrmartin: i unfortunately know next to nothing about redis, so spent a little time familiarizing myself with its configuration options. it looks like it's supposed to support paging out to disk | 18:07 |
| clarkb | the second rule of redis is that it will OOM your machine when it forks if you aren't careful with the memory you allocate to it | 18:07 |
| *** mestery has quit IRC | 18:07 | |
| fungi | clarkb: excellent pointers | 18:08 |
| *** patrickeast has quit IRC | 18:08 | |
| fungi | (sounds like java) | 18:08 |
| mrmartin | fungi: have you killed the old server? we could check the old redis config, and compare the settings | 18:08 |
| clarkb | fungi: yes but it forks to do that and unfortunately does enough munging of memory that copy on write pushes you over OOM killer limits | 18:08 |
| mrmartin | I just reused the openstackid redis here | 18:08 |
| cinerama | clarkb: well, without the updated wildfly in modules.env the class declarations for wildfly are going to be wrong | 18:08 |
| *** e0ne_ has quit IRC | 18:08 | |
| mrmartin | maybe it requires some spec configuration that was not written anywhere | 18:08 |
| clarkb | cinerama: ya, so now that 171790 merged 171791 should pass right? | 18:08 |
| fungi | mrmartin: good point. i'll diff the redis.conf between the old and new servers and see if there's something else we might have missed there | 18:09 |
| *** sdake has quit IRC | 18:09 | |
| *** spredzy is now known as spredzy|afk | 18:09 | |
| openstackgerrit | Merged openstack-infra/reviewstats: Drop myself from nova-core https://review.openstack.org/172160 | 18:09 |
| cinerama | clarkb: should do | 18:09 |
| clarkb | armax: fwiw we seem to fail a consistent 4-5 tests now which is less than the old 12-13 so I think we are making progress | 18:09 |
| clarkb | armax: the run for nova-net that I triggered is complete now, going to see if I can reproduce ssh failures now | 18:10 |
| armax | clarkb: agreed…the agent test failure I saw yesterday must have been a redherring | 18:10 |
| *** hashar has joined #openstack-infra | 18:10 | |
| *** mestery has joined #openstack-infra | 18:10 | |
| mrmartin | fungi: we are starting to build a staging server for ask.o.o because some new features / theme changes are in the queue | 18:11 |
| mrmartin | but need to solve the release and upgrade process, because this projects not lives under openstack-infra | 18:11 |
| mrmartin | so it will be a bit tricky, as we don't have those fancy jenkins jobs | 18:11 |
| *** sarob has quit IRC | 18:12 | |
| fungi | mrmartin: makes sense. we can certainly consider something like having pinned refs (tags? stable branches?) tracked for the production server and then development continuously deployed on the staging server | 18:13 |
| *** wenlock__ has joined #openstack-infra | 18:13 | |
| *** tqtran has joined #openstack-infra | 18:14 | |
| *** tosky has quit IRC | 18:15 | |
| mrmartin | fungi: it will be something like that | 18:15 |
| mrmartin | anyway, don't we have a project inside openstack the solves the problem of green / blue deployment models? | 18:16 |
| fungi | i thought that's what you were doing with the groups vs groups-dev sites? | 18:16 |
| mrmartin | something like that: http://docs.pivotal.io/pivotalcf/devguide/deploy-apps/blue-green.html but in an automatized form | 18:16 |
| mrmartin | I'm doing that for openstackid and groups, but openstackid is just a bunch of bash scripts, the groups a bunch of php | 18:17 |
| mrmartin | and it is doing hell similar things, and I like to avoid the situation to write a third very similar something for askbot | 18:17 |
| *** wenlock__ has quit IRC | 18:17 | |
| fungi | that blue-green workflow seems fairly complicated if we just want to demo newer code without needing access to the same database | 18:18 |
| mrmartin | it makes sense, if I like to use the same deployment model for staging servers as we are using for production ones | 18:18 |
| mrmartin | and of course it need to test the upgrade path too. | 18:19 |
| openstackgerrit | Davanum Srinivas (dims) proposed openstack/requirements: Allow Magnum to conform to global requirements process https://review.openstack.org/170830 | 18:19 |
| fungi | yeah, in theory the staging server could simply have a stale export of the database, or some generated sample data, or we could refresh it from production periodically if needed | 18:19 |
| mrmartin | it is starting to get complex when we have a db backend :) | 18:19 |
| *** _nadya_ has joined #openstack-infra | 18:19 | |
| mrmartin | which requires some upgrade, including schema or data | 18:19 |
| mrmartin | not talking about rollback options | 18:20 |
| clarkb | armax: this gets curiouser and curiouser | 18:20 |
| clarkb | armax: on the nova net env I held I can ssh from controller to compute VM and from compute VM to controller VM | 18:20 |
| armax | clarkb: that’s half good news | 18:20 |
| clarkb | armax: so I have a hunch these last few tests are failing for other reasons or a have a more subtle interaction with mtus | 18:20 |
| mrmartin | so it is great that heat can deploy an application, but what is the proper way of seamless upgrade? | 18:21 |
| *** wenlock__ has joined #openstack-infra | 18:21 | |
| fungi | armax: well, we could do like we do for gerrit and jenkins where we have dev servers and manually upgrade them, check them out to make sure they work, then manually upgrade production when we're happy with them | 18:21 |
| fungi | er, mrmartin ^ | 18:22 |
| fungi | sorry armax | 18:22 |
| *** YorikSar has joined #openstack-infra | 18:22 | |
| armax | fungi: np | 18:22 |
| openstackgerrit | Julia Kreger proposed openstack-infra/shade: Add update_machine method and operator unit test substrate https://review.openstack.org/171165 | 18:22 |
| openstackgerrit | Julia Kreger proposed openstack-infra/shade: Add test of OperatorCloud auth_type=None https://review.openstack.org/172165 | 18:22 |
| sc` | stupid question: would it make sense to have a centralized site (call it packages.o.o) where the distro packagers mirror their packaging efforts? | 18:22 |
| clarkb | mrmartin: fungi I would use ansible if that is your goal | 18:23 |
| *** ihrachyshka has quit IRC | 18:23 | |
| clarkb | since its orchestration of >1 cloud resource is actually pretty decent | 18:23 |
| armax | clarkb: ok, now I got my tunnel back | 18:23 |
| clarkb | sc`: you want to mirror the packages themselves or the repos that build the packages? | 18:23 |
| armax | clarkb: testing your ssh theory | 18:23 |
| sc` | one, then both | 18:24 |
| sc` | in my head at least :) | 18:24 |
| mrmartin | fungi: I don't like to make unemployment, but the manual upgrade tasks must be automatized somehow :D | 18:24 |
| clarkb | sc`: we already have an excellent place for hosting git repos | 18:24 |
| clarkb | sc`: so I think package maintainers could collaborate in gerrit for that | 18:24 |
| clarkb | whcih allows you to tie into Jenkins to build the packages from those repos | 18:24 |
| sc` | yeah but they're kinda fragmented at this point | 18:25 |
| clarkb | the only real bit missing is a place to host the packages | 18:25 |
| sc` | they use their own projects' infra | 18:25 |
| openstackgerrit | yolanda.robla proposed openstack-infra/nodepool: Different queues for slow tasks. https://review.openstack.org/170788 | 18:25 |
| clarkb | sc`: that isn't something that a website with a new name like packages.openstack.org will solve | 18:25 |
| fungi | mrmartin: i agree, but for things like gerrit and jenkins where upgrading them is touchy and we don't have a lot of control over changes to the process since it might evolve from release to release, sometimes automation is more effort than the manual process | 18:25 |
| sc` | supposed to be unmodified all around, but it gives the appearance of fragmentation | 18:25 |
| *** sdake has joined #openstack-infra | 18:25 | |
| clarkb | sc`: I think you have to drive it from the dev side first then we can work on hosting | 18:25 |
| sc` | clarkb: yeah, probably | 18:25 |
| clarkb | where dev side is package build repo in gerrit | 18:25 |
| mrmartin | fungi: I agree, but for those upfront services like web pages, openstackid, etc. I like to find a solution | 18:26 |
| *** sushilkm has left #openstack-infra | 18:26 | |
| fungi | mrmartin: i like to quote http://xkcd.com/1205/ as a reminder ;) | 18:26 |
| sc` | while i appreciate the packaging efforts thus far, i think them not being on openstack infra kind of dilutes things | 18:26 |
| *** tkelsey has quit IRC | 18:26 | |
| mrmartin | :D | 18:26 |
| *** ibiris is now known as ibiris_away | 18:26 | |
| mrmartin | I think worth the time, because those web-like services are multiplying. We have askbot now, but openstack.org will arrive soon, activity.o.o | 18:28 |
| sc` | but, you know, that's just, like, my opinion, man :) | 18:28 |
| *** marcusvrn has quit IRC | 18:28 | |
| clarkb | sc`: there is nothing stopping you (or anyone else) from pushing a package repo to gerrit, maintaining the package there, and using jenkins to build it | 18:29 |
| *** sdake_ has quit IRC | 18:29 | |
| fungi | clarkb: sc`: we started down that path with http://git.openstack.org/cgit/openstack-infra/zuul-packaging/ but it sort of lost steam | 18:30 |
| clarkb | sc`: the only missing bit is where do the build artifacts go, but I think we don't need to worry about that until some of the earlier steps happen | 18:31 |
| clarkb | armax: 3 out of 4 tests are volume/snapshot related. The fourth is shelve. | 18:31 |
| clarkb | armax: I am going to start debugging shelve first since I am already pretty familiar with it | 18:31 |
| *** kgiusti has left #openstack-infra | 18:31 | |
| fungi | mrmartin: yep, if they use consistent frameworks then automating upgrades for all of them is definitely worth the effort. if each is significantly different though, that's where it comes into question. i think the hard part is in orchestrating the different upgrade processes, less so finding a way to switch the servers around | 18:31 |
| armax | clarkb: it looks like I can ssh in my local env | 18:31 |
| *** SergeyLukjanov has quit IRC | 18:31 | |
| ctlaugh | Is there a way to configure Jenkins slave settings (such as JDK parameters, etc) for instances created by nodepool? | 18:32 |
| sc` | clarkb: yeah. i understand | 18:32 |
| *** SergeyLukjanov has joined #openstack-infra | 18:32 | |
| *** wenlock__ has quit IRC | 18:32 | |
| mrmartin | yeah, an existing Paas solution can solve this server switch story through an API, and really need to concentrate only on the switching / upgrade | 18:32 |
| clarkb | armax: thats good news, so ya I Think this is likely to be a new issue or a subtle play on an older issue. Should be fun to debug :) | 18:32 |
| armax | clarkb: how do you try to ssh? | 18:32 |
| clarkb | armax: I do ssh cirros@$floatingip | 18:32 |
| clarkb | armax: the cirros user passwd is cubswin:) | 18:33 |
| armax | clarkb: from which namespace? | 18:33 |
| clarkb | armax: I always operate in the root because that is where tempest is running | 18:33 |
| armax | or should it work in the base env? | 18:33 |
| asselin_ | ctlaugh, I just started looking at this...might be what you need: http://ci.openstack.org/jenkins-job-builder/wrappers.html#wrappers.inject | 18:33 |
| clarkb | armax: correct, tempest doesn't run in any special namespace its just running in the root ns | 18:33 |
| armax | clarkb: I can ssh from pretty much any namespace that has a route to the VM :) | 18:34 |
| armax | this is during manual testing | 18:34 |
| armax | clarkb: I am gonna figure out now, why tempest barf | 18:34 |
| armax | s | 18:34 |
| *** bhunter71 has quit IRC | 18:35 | |
| clarkb | well thats interesting I booted two nodes, test1 and test2. Put floating ip 1 on test1 and fip 2 on test2. Deleted test2 shelved test1 then unshelved test. And nova show shows both fips on test1 | 18:35 |
| armax | clarkb: I can ssh in either directions | 18:35 |
| *** sarob has joined #openstack-infra | 18:36 | |
| clarkb | neither fip1 or fip2 ping | 18:36 |
| armax | clarkb: ok, one issue at the time I guess :) | 18:36 |
| armax | clarkb: is this with Neutron, correct | 18:36 |
| armax | ? | 18:36 |
| clarkb | armax: no this is nova net | 18:36 |
| armax | clarkb: phew | 18:36 |
| armax | :) | 18:36 |
| clarkb | armax: I am starting there because I am already more familiar with that env | 18:36 |
| *** pblaho has quit IRC | 18:37 | |
| armax | clarkb: ok, I am gonna learn to walk before I can run…my env looks sane | 18:37 |
| armax | clarkb: the good news is that I used the d-g setup that you put together | 18:37 |
| armax | clarkb: and my devstack patch | 18:37 |
| clarkb | OH! I THink I know what the shelve bug is lol | 18:37 |
| armax | clarkb: it looks like tunnel is up etc | 18:37 |
| ctlaugh | asselin_: Is that for job-specific settings/control? I am looking to alter how the Jenkins slave gets launched -- i.e. Manage Jenkins | Manage Node | (select a node) | Configure | Advanced..., then configure JVM Options, etc. | 18:38 |
| ctlaugh | asselin_: It doesn't seem to be a system-wide setting in Jenkins, but node-specific. And, since nodepool causes the nodes to come and go, those settings don't stick... | 18:39 |
| *** _nadya_ has quit IRC | 18:39 | |
| asselin_ | ctlaugh, I see. Not sure how that's done. Perhaps as part of the nodepool image-prep scripts? | 18:40 |
| asselin_ | http://ci.openstack.org/nodepool/scripts.html | 18:40 |
| armax | clarkb: bollocks, forget what I said :) | 18:41 |
| ctlaugh | asselin_: ok, that looks promising... as long as I can find a way to set those node parameters via come command-line, config file, etc. Thank you for the pointer. | 18:42 |
| *** sarob has quit IRC | 18:43 | |
| *** bhunter71 has joined #openstack-infra | 18:50 | |
| armax | clarkb: have you seen https://jenkins04.openstack.org/job/check-tempest-dsvm-neutron-aiopcpu-full/23/console ? | 18:50 |
| armax | clarkb: we have a pass | 18:51 |
| *** doude has quit IRC | 18:51 | |
| clarkb | ya | 18:54 |
| *** dkehn has quit IRC | 18:57 | |
| *** spredzy|afk is now known as spredzy | 18:58 | |
| *** dizquierdo has quit IRC | 19:00 | |
| *** dmitryme has left #openstack-infra | 19:00 | |
| *** mrunge has quit IRC | 19:01 | |
| armax | clarkb: this is interesting | 19:01 |
| armax | clarkb: I think I can prove your mtu theory | 19:01 |
| *** _nadya_ has joined #openstack-infra | 19:01 | |
| clarkb | armax: oh good | 19:01 |
| armax | clarkb: but strangely enough only when I use the private ip ?!!? | 19:01 |
| armax | clarkb: it makes no sense, does it? | 19:02 |
| clarkb | also after the shelve unshelve my nova net br_pub is doa | 19:02 |
| openstackgerrit | yolanda.robla proposed openstack-infra/nodepool: Different queues for slow tasks. https://review.openstack.org/170788 | 19:02 |
| clarkb | armax: hrm private ip is all managed by neutron | 19:02 |
| armax | clarkb: yes | 19:02 |
| clarkb | armax: maybe neutron isn't appropriately changing the mtu to accomodate vxlan? | 19:02 |
| armax | clarkb: but the cirros vm has 1500 as mtu | 19:02 |
| armax | as soon as I flip to 1450 | 19:02 |
| armax | clarkb: I can ssh using the private ip | 19:03 |
| armax | clarkb: however, I can successfully ping the flaoting ip regardless | 19:03 |
| clarkb | ya ping comes in under mtu size so works | 19:03 |
| clarkb | ssh is a better test of this | 19:03 |
| armax | clarkb: sorry I mean ssh | 19:03 |
| armax | clarkb: ping works regardless | 19:03 |
| armax | clarkb: for obvious reasons ) | 19:03 |
| armax | :) | 19:03 |
| clarkb | new ovs frustration, I can't tcpdump the gre tunnel interface | 19:04 |
| clarkb | I'm not sure why people think this is an improvement over linux bridges | 19:04 |
| armax | clarkb: so I am a bit buffled | 19:04 |
| harlowja | qq, for https://github.com/stackforge/cloud-init i want to create a branch for the 0.7.x series; butttt i am getting this from gerrit | 19:07 |
| harlowja | ! [remote rejected] 0.7.x -> 0.7.x (prohibited by Gerrit) | 19:07 |
| *** mmedvede has quit IRC | 19:07 | |
| harlowja | any ideas? | 19:07 |
| harlowja | some kind of ACL missing? | 19:07 |
| clarkb | harlowja: you cannot force push which is required to create branches via git | 19:08 |
| clarkb | harlowja: try creating it in the web ui | 19:08 |
| clarkb | harlowja: if that fails then you need an acl update | 19:08 |
| openstackgerrit | Merged openstack-infra/project-config: python-saharaclient: make the py3 gate voting https://review.openstack.org/172076 | 19:08 |
| harlowja | clarkb k | 19:09 |
| AJaeger | harlowja: looks like you need an ACL update | 19:09 |
| harlowja | ya, seems read-only @ https://review.openstack.org/#/admin/projects/stackforge/cloud-init,branches | 19:09 |
| * AJaeger just checked gerrit/acls/stackforge/cloud-init.config | 19:09 | |
| *** mrunge has joined #openstack-infra | 19:09 | |
| harlowja | thx | 19:09 |
| *** e0ne has joined #openstack-infra | 19:10 | |
| *** erikmwilson has quit IRC | 19:11 | |
| openstackgerrit | Joshua Harlow proposed openstack-infra/project-config: Allow access to stable branches https://review.openstack.org/172179 | 19:12 |
| harlowja | ^ might not be right, let me know | 19:12 |
| fungi | git-review seems to attract its share of first-time openstack contributors. i've seen two "welcome new contributor" comments on changes proposed to that repo in the past week or so | 19:12 |
| armax | clarkb: so where does this leave us? I got two passes in a row on https://review.openstack.org/#/c/171809/ and my local setup seems ot be happy even when I exercise it with tempest | 19:12 |
| reed | fungi, when you have time, could you check that askbot can send emails? https://bugs.launchpad.net/openstack-community/+bug/1442317 | 19:13 |
| openstack | Launchpad bug 1442317 in OpenStack Community Project "Validation emails not being sent" [Undecided,New] | 19:13 |
| openstackgerrit | Joshua Harlow proposed openstack-infra/project-config: Allow cloud-init-core access to stable branches https://review.openstack.org/172179 | 19:13 |
| openstackgerrit | Levi Blackstone proposed openstack-infra/project-config: Add gating jobs for stacktach-winchester https://review.openstack.org/172180 | 19:13 |
| fungi | reed: i'll see what the logs tell me, gladly | 19:13 |
| fungi | reed: ahh, yep. we have a configuration issue | 19:14 |
| *** amitgandhinz has quit IRC | 19:14 | |
| *** amitgandhinz has joined #openstack-infra | 19:15 | |
| fungi | reed: ask seems to want to send e-mail as "openstack@ask.openstack.org" which can't receive e-mail | 19:15 |
| armax | clarkb: the bad news is that n-net aiopcpu seems to be failing | 19:15 |
| *** primeministerp has quit IRC | 19:15 | |
| armax | clarkb: pretty consistently | 19:15 |
| fungi | reed: what from address should it be using? or should we set up an mx record for ask.openstack.org to somewhere we can create an "openstack" mailbox to receive (and probably /dev/null) any replies? | 19:16 |
| reed | fungi, probably that is a config I can change from the UI? | 19:16 |
| fungi | reed: i don't know. just trying to outline our options | 19:16 |
| fungi | reed: another option would be to work out how to tell exim to allow e-mail from that address without verifying it can receive messages, but that seems like an iffy choice | 19:17 |
| clarkb | armax: I think we need to debug nova net | 19:17 |
| clarkb | armax: my env is currently hosed after the shelve unshelve thing. I don't see packets cross the ovs gre bridge at all | 19:17 |
| clarkb | armax: but routes are fine so not sure what is going on | 19:17 |
| clarkb | armax: also I can pass traffic over the unencapsulated interface that the gre bridge uses | 19:18 |
| reed | fungi, can you update the bug please? I'll assign it to evgeny | 19:18 |
| fungi | reed: sure. i'm still analyzing but i'll add something there | 19:18 |
| *** ivar-lazzaro has quit IRC | 19:19 | |
| fungi | reed: interestingly it's not completely broken because it e-mailed me about a question earlier today | 19:19 |
| *** _nadya_ has quit IRC | 19:19 | |
| armax | clarkb: it’s that only those 4 tests fail for n-net | 19:20 |
| *** dprince has quit IRC | 19:20 | |
| *** amitgandhinz has quit IRC | 19:20 | |
| *** amitgandhinz has joined #openstack-infra | 19:20 | |
| clarkb | armax: ya and I think shelve fails first | 19:20 |
| *** shashankhegde has quit IRC | 19:20 | |
| armax | clarkb: maybe they are the ones where the vm lands on the subnode? | 19:20 |
| clarkb | armax: so I have a semi theory that shelve runs and breaks networking then the tests that need to ssh after fail | 19:20 |
| armax | clarkb: oh | 19:20 |
| armax | clarkb: that’s a theory | 19:21 |
| openstackgerrit | Spencer Krum proposed openstack-infra/system-config: Bring in module to manage update-alternatives https://review.openstack.org/171556 | 19:21 |
| *** Longgeek_ has quit IRC | 19:22 | |
| *** Krinkle is now known as Krinkle|detached | 19:22 | |
| *** [HeOS] has joined #openstack-infra | 19:23 | |
| *** isviridov has joined #openstack-infra | 19:25 | |
| clarkb | armax: I have br_pub 172.24.4.1 <-- gre --> 172.24.4.2 br_pub if I try to ping 172.24.4.2 from 172.24.4.1 I see arp who has messages on 172.24.4.1's br_pub but not on 172.24.4.2's br_pub | 19:25 |
| clarkb | armax: ping then reports icmp destination host unreachable to 172.24.4.1 | 19:26 |
| clarkb | armax: any idea why that ovs gre bridge would break like that? | 19:26 |
| fungi | reed: it looks like it sends some messages successfully as root@ask.openstack.org so we could work around it by aliasing openstack to root in /etc/aliases on the server for now | 19:26 |
| reed | ok, thanks | 19:26 |
| armax | clarkb: what does ovs-vsctl show tell you? | 19:27 |
| fungi | reed: but ideally we should consider how best to get it using a consistent address we want it to use rather than arbitrary local usernames at that server's fqdn | 19:27 |
| armax | clarkb: next thing to try is ovs-dpctl to see why the flows are broken | 19:27 |
| clarkb | ugh is that why tcpdump doesn't work on the gre interfaces | 19:27 |
| clarkb | clearly I should learn myself an ovs but its not presenting itself well | 19:28 |
| *** isviridov has quit IRC | 19:28 | |
| clarkb | armax: ovs-vsctl show still shows the bridge as being there with a gre interface | 19:28 |
| armax | clarkb: ok and both remote and local ip are fine, correct? | 19:29 |
| clarkb | armax: ya I can ping over the addresses given by rackspace | 19:29 |
| armax | clarkb: we could try and do ovs-dpctl dump-flows and grep for either IP’s or MAC's | 19:29 |
| clarkb | which is what the gre tunnel runs over | 19:29 |
| *** weshay has quit IRC | 19:29 | |
| armax | do see what action is being enforced | 19:29 |
| armax | while you’re doing the ping | 19:29 |
| armax | that should be reveiling | 19:29 |
| clarkb | armax: actions:drop | 19:31 |
| reed | fungi, agreed | 19:31 |
| armax | clarkb: ah | 19:31 |
| armax | clarkb: there you! | 19:31 |
| armax | og | 19:31 |
| armax | go | 19:31 |
| reed | let's update the ticket with the details and i'll sort it out with evgeny | 19:31 |
| armax | clarkb: now we’d need to understand why! | 19:31 |
| clarkb | armax: indeed, any idea where to look next? | 19:32 |
| clarkb | would iptables affect the ovs actions? | 19:32 |
| armax | clarkb: ovs-ofctl should be our next ally | 19:32 |
| armax | clarkb: don’t think so | 19:32 |
| clarkb | we should be wide open between the two nodes with iptables though | 19:32 |
| clarkb | armax: thanks /me reads more manpages | 19:32 |
| armax | can you post the whole row? | 19:32 |
| armax | clarkb: where you saw the drop? | 19:32 |
| *** AJaeger has quit IRC | 19:32 | |
| *** pblaho has joined #openstack-infra | 19:33 | |
| clarkb | armax: http://paste.openstack.org/show/201479/ its the only flow too | 19:34 |
| clarkb | armax: the source mac matches the br_pub interfaces so its going into the correct bridge | 19:35 |
| armax | try sudo ovs-ofctl dump-flows br_pub | 19:36 |
| clarkb | so routes are happy | 19:36 |
| *** esker has joined #openstack-infra | 19:36 | |
| armax | clarkb: but I am trying to remember the command that would tell us ethe culprit… | 19:36 |
| armax | clarkb: hang on | 19:36 |
| openstackgerrit | Joshua Harlow proposed openstack-infra/project-config: Allow access to 0.7.x branch(es) https://review.openstack.org/172179 | 19:36 |
| clarkb | armax: http://paste.openstack.org/show/201480/ | 19:36 |
| clarkb | armax: hrm I do notice that br_pub and br_flat both have states of UNKNOWN | 19:38 |
| clarkb | maybe they need to be explicitly up? | 19:38 |
| armax | clarkb: um | 19:38 |
| *** sarob has joined #openstack-infra | 19:39 | |
| armax | clarkb: what if you do ofctl without br_pub? | 19:40 |
| clarkb | dump-flows takes a switch argument | 19:40 |
| *** sarob has quit IRC | 19:40 | |
| *** sarob has joined #openstack-infra | 19:40 | |
| armax | oops sorry I got confused with dpctl | 19:41 |
| armax | clarkb: nm | 19:41 |
| *** mrunge has quit IRC | 19:42 | |
| *** bruff has joined #openstack-infra | 19:42 | |
| *** erikmwilson_ is now known as erikmwilson | 19:42 | |
| bruff | http://status.openstack.org/zuul/ | 19:44 |
| armax | clarkb: can you post also the content of ovs-vsctl show? | 19:45 |
| *** mmedvede has joined #openstack-infra | 19:45 | |
| harlowja | clarkb does https://review.openstack.org/#/c/172179/ seem ok with u; only really want a 0.7.x branch (and tags there eventually) | 19:45 |
| harlowja | never know if i get that ACL stuff right, ha | 19:45 |
| *** xyang1 has joined #openstack-infra | 19:46 | |
| *** johnchalekson has quit IRC | 19:47 | |
| clarkb | armax: http://paste.openstack.org/show/201493/ btoh nodes are there as long as an ip addr show on the interfaces that we tunnel over | 19:48 |
| *** e0ne has quit IRC | 19:49 | |
| *** ivar-lazzaro has joined #openstack-infra | 19:49 | |
| *** dims__ is now known as dims | 19:50 | |
| *** Sukhdev has quit IRC | 19:52 | |
| clarkb | armax: what shelve is doing is it moves a VM from one host to the other offline. when it brings the node back up it reinstalls the floating ip on the new compute node | 19:52 |
| clarkb | armax: could ovs be having a fit because that IP is moving? I wouldn't expect it to bceause IPs get new source macs all the time | 19:52 |
| clarkb | (see dhcp) | 19:53 |
| openstackgerrit | Matthew Treinish proposed openstack-infra/subunit2sql: Add db api methods to delete old runs and test_runs https://review.openstack.org/172190 | 19:53 |
| armax | clarkb: let’s try sudo ovs-vsctl list interface now | 19:55 |
| armax | clarkb: we’d need to identify what’s the port where the drop happens | 19:55 |
| armax | clarkb: do you still have the ping going? | 19:55 |
| clarkb | armax: yes ping is still running | 19:55 |
| *** ayoung-lunch is now known as ayoung | 19:55 | |
| *** ociuhandu has quit IRC | 19:57 | |
| *** otter768 has joined #openstack-infra | 19:57 | |
| clarkb | armax: http://paste.openstack.org/show/201496/ | 19:59 |
| *** otter768 has quit IRC | 20:01 | |
| armax | clarkb: not sure if I am mistaken, but I don’t seem to find the port affected by the flows | 20:03 |
| clarkb | armax: ya I don't see it either | 20:04 |
| armax | clarkb: btw http://paste.openstack.org/show/201479/ is from the sender correct? | 20:04 |
| clarkb | armax: corect that paste is from the first host (The sender) | 20:04 |
| *** tnovacik has joined #openstack-infra | 20:04 | |
| clarkb | I am going to make a small change to how nova net uses the bridge in functions.sh | 20:05 |
| clarkb | instead of telling nova net to use the bridges directly (whihc maybe it explodes the bridge doing) I am going to add a veth pair, one side for nova one side on the bridge | 20:05 |
| *** davideagnello has quit IRC | 20:06 | |
| armax | clarkb: it’s worth trying | 20:06 |
| *** davideagnello has joined #openstack-infra | 20:07 | |
| *** tonytan4ever has quit IRC | 20:07 | |
| *** bruff has left #openstack-infra | 20:11 | |
| armax | clarkb: actually ovs-dpctl show should tell us what port 3 is | 20:12 |
| *** soren_ has joined #openstack-infra | 20:13 | |
| *** bradjones has quit IRC | 20:13 | |
| openstackgerrit | Clark Boylan proposed openstack-infra/devstack-gate: DO NOT MERGE testing overlay network options https://review.openstack.org/171714 | 20:13 |
| clarkb | ok thats should run experimental tests | 20:14 |
| clarkb | let me see what ovs-dpctl says | 20:14 |
| *** tonytan4ever has joined #openstack-infra | 20:14 | |
| *** signed8bit has joined #openstack-infra | 20:14 | |
| clarkb | armax: http://paste.openstack.org/show/201505/ its the br_pub port | 20:15 |
| armax | ok | 20:15 |
| armax | so it’s really the right port where the packet must go | 20:16 |
| clarkb | yup I it seems to be starting off correctly | 20:16 |
| clarkb | armax: also one difference between this in and the neutron setup is we have two gre tunnels | 20:16 |
| clarkb | armax: maybe we need extra options to make that work correctly? | 20:17 |
| armax | what do you mean? | 20:17 |
| armax | two gre tunnels? | 20:17 |
| clarkb | yes nova net requires two l2s. The "flat" network for vm internal traffic and the "pub" network where you get out to the internet | 20:17 |
| clarkb | so we have one gre bridge for flat and another for pub | 20:17 |
| *** bradjones has joined #openstack-infra | 20:17 | |
| clarkb | we tried doing one at one point but if you do that nova's ebtables gets really made at you | 20:18 |
| *** wenlock__ has joined #openstack-infra | 20:21 | |
| *** mwagner_lap has quit IRC | 20:22 | |
| armax | clarkb: I don’t htink that sohuld be a problem | 20:23 |
| *** achanda has quit IRC | 20:24 | |
| armax | clarkb: I am trying t oremember how to use ovs-appctl, but it doesn’t seem to work for me | 20:24 |
| armax | clarkb: bear with me | 20:24 |
| *** luqas has joined #openstack-infra | 20:27 | |
| asselin_ | just started seeing juno devstack failing ERROR: openstackclient.shell Exception raised: (oslo.utils 1.2.1 (/usr/local/lib/python2.7/dist-packages), Requirement.parse('oslo.utils<1.5.0,>=1.4.0'), set(['python-neutronclient'])) http://paste.openstack.org/show/201515/ | 20:28 |
| asselin_ | sigh....I see python-neutronclient just released... | 20:29 |
| asselin_ | mestery, ^^ is that related to python-neutronclient? | 20:29 |
| mestery | asselin_: Yes, I'm replying on this review here: https://review.openstack.org/#/c/172150/ | 20:30 |
| mestery | asselin_: We need that to merge to cap things in Juno, I wrote a comment there, I do see sdague's point and hope we didn't screw things up too badly (amotoki and I). | 20:30 |
| openstackgerrit | Matthew Treinish proposed openstack-infra/subunit2sql: Add a cli subcommand for deleting old rows https://review.openstack.org/172200 | 20:30 |
| asselin_ | mestery, ok thanks...glad it's in progress | 20:31 |
| mtreinish | pcrews: ^^^ although I haven't actually tested that stack, so chances are it doesn't work | 20:31 |
| *** marcusvrn has joined #openstack-infra | 20:31 | |
| *** luqas has quit IRC | 20:31 | |
| *** wenlock__ has quit IRC | 20:32 | |
| * pcrews is looking at that patch :) | 20:32 | |
| *** cody-somerville has quit IRC | 20:32 | |
| *** tsg has joined #openstack-infra | 20:33 | |
| *** baoli has joined #openstack-infra | 20:36 | |
| armax | clarkb: try ovs-appctl ofproto/trace br-pub | 20:37 |
| armax | clarkb: that should tell use why ovs is treating the packets the way it is | 20:38 |
| *** hdd has joined #openstack-infra | 20:38 | |
| *** ivar-lazzaro has quit IRC | 20:38 | |
| clarkb | it says bad flow syntax | 20:39 |
| *** doug-fish has left #openstack-infra | 20:40 | |
| *** shashankhegde has joined #openstack-infra | 20:41 | |
| openstackgerrit | Armando Migliaccio proposed openstack-infra/devstack-gate: ***DO NOT MERGE*** Flip DVR bit to collect experimental run datapoints https://review.openstack.org/171809 | 20:41 |
| *** shashankhegde has quit IRC | 20:42 | |
| *** shashankhegde has joined #openstack-infra | 20:42 | |
| clarkb | armax: http://paste.openstack.org/show/201518/ gave it in_port=3 | 20:42 |
| *** e0ne has joined #openstack-infra | 20:42 | |
| *** e0ne has quit IRC | 20:43 | |
| armax | clarkb: ok, it says no input bundle, dropping | 20:44 |
| armax | clarkb: so I guess the tunnel gets busted somehow | 20:44 |
| *** adalbas has quit IRC | 20:44 | |
| clarkb | so maybe nova net is affecting it somehow | 20:45 |
| clarkb | my latest patch should test that I think | 20:45 |
| armax | clarkb: ok | 20:46 |
| *** andreykurilin_ has joined #openstack-infra | 20:46 | |
| armax | clarkb: we had another semi-successful run with neutron in https://review.openstack.org/#/c/171809/ | 20:46 |
| armax | clarkb: we’d need to chase why the agent test is racey but it looks like the ssh tests are fairly happy | 20:47 |
| armax | clarkb: your latest additions | 20:47 |
| armax | I wonder if we should pull tehm in in the parent patch | 20:48 |
| clarkb | if they work I eill | 20:48 |
| clarkb | I just had this change ready to go | 20:48 |
| *** baoli has quit IRC | 20:49 | |
| armax | we’d need to figure out what’s the effect on devstack though | 20:49 |
| *** pcrews has quit IRC | 20:49 | |
| *** baoli has joined #openstack-infra | 20:49 | |
| *** smccully has quit IRC | 20:49 | |
| *** achanda has joined #openstack-infra | 20:53 | |
| *** pcrews has joined #openstack-infra | 20:53 | |
| *** sdake has quit IRC | 20:53 | |
| openstackgerrit | Stephanie Miller proposed openstack-infra/system-config: Add infra setup for zanata https://review.openstack.org/169074 | 20:54 |
| *** bswartz has quit IRC | 20:57 | |
| *** stevemar has quit IRC | 20:58 | |
| pleia2 | cinerama: where does the mysql username+password come from? | 20:59 |
| cinerama | pleia2: oh whoops, i knew i was forgetting something | 20:59 |
| pleia2 | cinerama: just making sure *I* wasn't missing something :) | 20:59 |
| *** crc32 has joined #openstack-infra | 21:01 | |
| openstackgerrit | Stephanie Miller proposed openstack-infra/system-config: Add infra setup for zanata https://review.openstack.org/169074 | 21:01 |
| cinerama | pleia2: i think the defaults are fine otherwise but we do need the password | 21:01 |
| clarkb | pleia2: cinerama are we buying a cert for the dev server or just going to use snakeoil certs? I ask because the site.pp appears to use hiera instead of on disk snakeoil certs | 21:03 |
| cinerama | i honestly don't know | 21:03 |
| cinerama | i am going to defer to pleia2 here | 21:03 |
| pleia2 | snakeoil is fine | 21:04 |
| clarkb | pleia2: fwiw I did't approve https://review.openstack.org/#/c/166109/4 bceause you all mentioned babysitting so I wanted to know what babysitting you had in mind | 21:04 |
| *** mwagner_lap has joined #openstack-infra | 21:04 | |
| clarkb | just checking the results on the slave node? | 21:04 |
| pleia2 | clarkb: mostly just making sure it's successful and doesn't mess up the proposal slave in any way | 21:04 |
| clarkb | ok /me goes to approve | 21:05 |
| *** tiswanso has quit IRC | 21:05 | |
| *** hdd has quit IRC | 21:05 | |
| *** emagana has joined #openstack-infra | 21:06 | |
| *** rfolco has quit IRC | 21:07 | |
| clarkb | armax: the shelve instance test failed on my latest patchset so this may not be an improvement | 21:08 |
| *** aysyd has quit IRC | 21:08 | |
| clarkb | should wait for tempest to give us tracebacks though | 21:08 |
| *** esker has quit IRC | 21:10 | |
| armax | clarkb: I wonder if this demands for a dual-proged strategy where we set up nova and neutron differently :( | 21:10 |
| clarkb | armax: it might | 21:10 |
| openstackgerrit | Matt Riedemann proposed openstack-infra/elastic-recheck: Add query for bug 1442367 https://review.openstack.org/172211 | 21:10 |
| openstack | bug 1442367 in python-neutronclient "2.4.0 release destroys grenade since stable/juno isn't capped properly" [Critical,Confirmed] https://launchpad.net/bugs/1442367 | 21:10 |
| clarkb | would at least like to understand why before I Give in though :) | 21:10 |
| armax | clarkb: not sure I personally have the chops to understand what’s really going on | 21:10 |
| armax | clarkb: true | 21:10 |
| armax | clarkb: going to get some lunch, bbiab | 21:11 |
| *** jcoufal has joined #openstack-infra | 21:12 | |
| pleia2 | cinerama: oof, put comment in the wrong patchset (9 instead of 10) but I left an example of how we use snakeoil certs in jenkins-dev | 21:12 |
| *** ayoung has quit IRC | 21:12 | |
| *** arxcruz has quit IRC | 21:13 | |
| openstackgerrit | Merged openstack-infra/system-config: Use zanata::client in proposal slave https://review.openstack.org/166109 | 21:14 |
| openstackgerrit | Stephanie Miller proposed openstack-infra/system-config: Add infra setup for zanata https://review.openstack.org/169074 | 21:14 |
| clarkb | mriedem: if you just want xenserver ci to stop leaving -1's nova has control of that | 21:15 |
| clarkb | mriedem: or do you want all comments to be disabled? | 21:15 |
| mriedem | clarkb: they told me to email the ML | 21:15 |
| pleia2 | cinerama: so since we're never going to use real certs on the dev server, you can remove the hiera lookups in the site.pp too | 21:15 |
| clarkb | mriedem: they is who? | 21:16 |
| mriedem | mika | 21:16 |
| mriedem | mikal | 21:16 |
| openstackgerrit | Stephanie Miller proposed openstack-infra/system-config: Add infra setup for zanata https://review.openstack.org/169074 | 21:16 |
| openstackgerrit | Merged openstack-infra/elastic-recheck: Add query for bug 1442367 https://review.openstack.org/172211 | 21:18 |
| openstack | bug 1442367 in python-neutronclient "2.4.0 release destroys grenade since stable/juno isn't capped properly" [Critical,Confirmed] https://launchpad.net/bugs/1442367 - Assigned to Akihiro Motoki (amotoki) | 21:18 |
| pleia2 | cinerama: doh, I didn't notice the bup section until now :\ I think we want to wait on configuring backups too | 21:19 |
| cinerama | pleia2: okay. | 21:19 |
| pleia2 | still haven't finished setting up backups for ask.o.o, need to make sure we have space for everything | 21:19 |
| pleia2 | and don't want to hold off getting this server up for that :) | 21:19 |
| cinerama | some of the dev sites do use bup fwiw | 21:20 |
| clarkb | they shouldn't | 21:20 |
| clarkb | I think we are trying to not bup dev sites | 21:20 |
| pleia2 | I think only jenkins-dev does (review-dev was removed recently after being disabled for a while) | 21:21 |
| *** dims_ has joined #openstack-infra | 21:21 | |
| clarkb | we should probably stop jenkins-dev at some point too | 21:22 |
| *** marcusvrn has quit IRC | 21:24 | |
| *** dims has quit IRC | 21:24 | |
| *** dims_ has quit IRC | 21:26 | |
| jesusaurus | clarkb 4 neutron ptl | 21:26 |
| *** hashar has quit IRC | 21:26 | |
| clarkb | heh | 21:27 |
| *** ldnunes has quit IRC | 21:28 | |
| bkero | seconded | 21:29 |
| openstackgerrit | Stephanie Miller proposed openstack-infra/system-config: Add infra setup for zanata https://review.openstack.org/169074 | 21:31 |
| fungi | yeah, jenkins-dev was originally being backed up as a test | 21:31 |
| fungi | before we added the module to any production systems | 21:31 |
| fungi | so we could test restores safely and whatnot | 21:31 |
| *** r-daneel has joined #openstack-infra | 21:31 | |
| clarkb | fungi: ya, and now that its all working its just noise on the backup server | 21:32 |
| fungi | totally | 21:33 |
| fungi | zaro: does this look like what you're expecting? http://logs.openstack.org/32/171732/2/check/check-gerrit-buck/4a98ac6/console.html#_2015-04-09_20_08_13_354 | 21:35 |
| *** mancdaz has quit IRC | 21:35 | |
| clarkb | russellb: your name shows up in ovs commit logs so I am going to bug you :) | 21:35 |
| fungi | that's usually my preferred criteria for annoying people | 21:35 |
| *** dkranz has quit IRC | 21:36 | |
| pleia2 | cinerama: looks good :) we can let tests finished before someone +A but I left my +2 and am going to grab late lunch (not sure how it got this late, argh meetings) | 21:36 |
| clarkb | russellb: for multinode nova network testing we are trying to setup two overlay networks with ovs and gre so that the compute nodes can be on the same l2 for the "flat" network and the "pub" network | 21:36 |
| zaro | fungi: yep, looks good! | 21:36 |
| clarkb | russellb: the overlay is necessary because our clouds don't give our test envs isolated l2s or garuntee that they weill be on the same l2 | 21:36 |
| fungi | zaro: cool, i'll go ahead and approve the pending changes for that branch then | 21:36 |
| *** ivar-lazzaro has joined #openstack-infra | 21:37 | |
| clarkb | russellb: trouble is at some point the ovs interface decides to drop arp packets | 21:37 |
| clarkb | russellb: and nothing works once that happens | 21:37 |
| russellb | i was *just* about to shut down for the day ... | 21:37 |
| russellb | but it works at first, and just stops later? | 21:37 |
| clarkb | russellb: correct | 21:37 |
| *** mancdaz has joined #openstack-infra | 21:38 | |
| clarkb | russellb: I can bug you tomorrow and pick it up then if it helps :) | 21:38 |
| russellb | i guess i'd dump the switch config and flow tables at the beginning when it works, and then later when it's busted, to see what is different | 21:38 |
| cinerama | pleia2: ok sounds good | 21:38 |
| russellb | could be "fun" to help debug | 21:38 |
| russellb | but yeah, can't today | 21:38 |
| adam_g | clarkb, what are the symptoms? | 21:38 |
| clarkb | russellb: ok | 21:38 |
| russellb | could also see if anyone in #openstack-neutron is interested in jumping in | 21:38 |
| clarkb | russellb: ya armax has been helping a bunch | 21:39 |
| russellb | awesome | 21:39 |
| clarkb | but said earlier that this is getting hairy | 21:39 |
| russellb | i'm no pro at that level yet anyway :) | 21:39 |
| russellb | i'm out ttyl | 21:39 |
| clarkb | adam_g: tempest fails because ssh times out. If I get on the node and ping from one hypervisor to the other I get destination unreachable | 21:39 |
| clarkb | adam_g: then I tcpdump and find arp who has goes no where | 21:39 |
| clarkb | adam_g: and a ovs-ofctl dump-flow shows my arp who has being dropped by ovs | 21:40 |
| fungi | clarkb: when you have a sec, can i get a rubber stamp on https://review.openstack.org/172105 and then i'll reenable puppet agent on ask.o.o | 21:40 |
| *** gyee has joined #openstack-infra | 21:41 | |
| clarkb | fungi: stamped | 21:41 |
| *** ivar-lazzaro has quit IRC | 21:41 | |
| fungi | danke | 21:41 |
| clarkb | I also have to say it is incredibly frustrating that I can't tcpdump this stuff | 21:41 |
| adam_g | clarkb, oh, weird. thats a bit different than the weirdness i was seeing when i was messing with that multihost setup a while back. for me, the GRE between the two hosts was causing fragmentation and MTUs needed to be dropped | 21:41 |
| clarkb | adam_g: ya so the old gre setup with linux bridge seems to work fine | 21:42 |
| *** dustins has quit IRC | 21:42 | |
| clarkb | adam_g: but playing with ovs to accomodate neutron and dvr with a single set of overlay code | 21:42 |
| *** radez is now known as radez_g0n3 | 21:42 | |
| fungi | clarkb: is there another kernel module you need to provide a filter interface for tcpdump on the bridge? | 21:42 |
| clarkb | fungi: maybe? I honestly don't know. OVS is mostly new to me and having now used it for 2 days I will probably only use it when I want to have a bad time | 21:43 |
| clarkb | there are basically zero docs | 21:43 |
| fungi | i always get confused between bsd's bpf model and how the linux kernel allows binding for sniffers | 21:43 |
| clarkb | existing network tools don't work with it | 21:43 |
| clarkb | and for some reason it has decided it must drop my arp packages | 21:43 |
| clarkb | er packets | 21:44 |
| clarkb | adam_g: also ovs and linux bridge handle fragmentation differently | 21:45 |
| fungi | clarkb: oh... for ovs you may need port or vlan mirroring enabled? | 21:45 |
| clarkb | adam_g: which I thought was part of the problem initially but I lowered MTUs and its still a problem | 21:45 |
| *** Sukhdev has joined #openstack-infra | 21:45 | |
| fungi | e.g. configure it to mirror all frames for a particular vlan to a specific vswitch port | 21:45 |
| clarkb | fungi: I can't tcpdump ovs interfaces because they don't exist as linux devices | 21:45 |
| clarkb | adam_g: sorry lowered MTUs and ssh still times out, after closer inspection its because arp is dropped | 21:46 |
| openstackgerrit | Victor Stinner proposed openstack-infra/project-config: oslo.messaging: enable Python 3.4 check https://review.openstack.org/172221 | 21:46 |
| *** mrmartin has quit IRC | 21:48 | |
| fungi | poking around, it looks like the suggestion is to bind an actual kernel-recognized interface to the vswitch as a switchport, add port mirroring to that interface, and then stick tcpdump on it | 21:50 |
| fungi | still hunting good examples though | 21:51 |
| *** andreykurilin_ has quit IRC | 21:51 | |
| clarkb | it might be easiest to use the old gre linux bridge stuff and just make br-ex with a port from that for neutron | 21:52 |
| ianw | before i figure something out, does anyone know of a way to list the recent votes of a user. i'm pretty sure "Citrix XenServer CI" is broken | 21:52 |
| adam_g | clarkb, i feel like jamespage hit something very similar at one point doing openstack-on-openstack testing a while back | 21:52 |
| fungi | ianw: mriedem just dropped them from the nova-ci group for that reason | 21:52 |
| clarkb | ianw: ya its something like label:verified=-1,jenkins age:-2days | 21:53 |
| clarkb | I am going to try the linux bridge idea | 21:53 |
| fungi | clarkb: http://www.yet.org/2014/09/openvswitch-troubleshooting/ | 21:54 |
| fungi | adding a veth0 interface to the ovs and setting up port mirroring in it | 21:54 |
| fungi | example command lines in the "open vswitch mirroring" section of that article | 21:55 |
| clarkb | thats pretty amazing | 21:56 |
| clarkb | it takes 5 commends to be able to tcpdump one of which is super gigantic | 21:56 |
| fungi | "this is how you sniff an ovs" | 21:56 |
| fungi | yah, srsly | 21:56 |
| fungi | "this is a virtual switch, so we should make connecting a sniffer to it just as complicated as trying to sniff on a real physical switch" | 21:57 |
| *** weshay has joined #openstack-infra | 21:58 | |
| *** otter768 has joined #openstack-infra | 21:58 | |
| fungi | i can pretty much read that as the junos/catos/ios/nxos commands i used to use to set up port mirroring, along with the physical cabling to connect it to the machine where i'm running the sniffer | 21:59 |
| clarkb | I am a big -1 on using ovs for infra cloud now fwiw | 21:59 |
| fungi | just because you can make common things easy, doesn't mean you should! | 21:59 |
| openstackgerrit | Matthew Treinish proposed openstack-infra/devstack-gate: DO NOT MERGE: Test Using secure delete to verify bug https://review.openstack.org/172225 | 21:59 |
| fungi | i think most network admins suffer from stockholm syndrome | 22:00 |
| openstackgerrit | Merged openstack-dev/hacking: Updated from global requirements https://review.openstack.org/166425 | 22:01 |
| ianw | fungi: thanks, i really should catch up on openstack-dev first thing in the morning before i ever say anything :) | 22:02 |
| *** otter768 has quit IRC | 22:03 | |
| *** soren_ has quit IRC | 22:03 | |
| *** sarob has quit IRC | 22:03 | |
| fungi | ianw: also doughnuts | 22:03 |
| fungi | priorities, man... priorities | 22:03 |
| *** emagana has quit IRC | 22:03 | |
| *** dboik_ has joined #openstack-infra | 22:06 | |
| ianw | so i replied to the thread, but if someone could flick it off in https://review.openstack.org/#/admin/groups/507,members that would be awesome | 22:06 |
| *** Swami has quit IRC | 22:07 | |
| *** pc_m has quit IRC | 22:07 | |
| *** sdake has joined #openstack-infra | 22:07 | |
| fungi | sdague: dtroyer: ^ | 22:07 |
| fungi | mtreinish: ^ | 22:07 |
| fungi | any of the three of them should be able to | 22:07 |
| mtreinish | fungi: oh turn off citrix | 22:09 |
| mtreinish | yeah one sec | 22:09 |
| mtreinish | fungi: just delete it from that group? | 22:09 |
| fungi | we're trying to keep the individual project leadership weeding their own gardens when possible | 22:09 |
| fungi | mtreinish: yep | 22:09 |
| *** dboik has quit IRC | 22:09 | |
| mtreinish | ianw, fungi: ok deleted it | 22:09 |
| openstackgerrit | Merged openstack-dev/hacking: Fix string formatting in example https://review.openstack.org/168269 | 22:10 |
| ianw | mtreinish: thanks, i've also updated their 3rd-party wiki page https://wiki.openstack.org/wiki/ThirdPartySystems/XenServer_CI | 22:10 |
| *** dboik_ has quit IRC | 22:10 | |
| *** sdake_ has joined #openstack-infra | 22:10 | |
| mtreinish | I probably should do that on tempest too | 22:10 |
| *** Guest77301 has joined #openstack-infra | 22:11 | |
| openstackgerrit | Khai Do proposed openstack-infra/system-config: upgrade review-dev to Gerrit ver 2.10 https://review.openstack.org/172226 | 22:11 |
| zaro | fungi: ^ | 22:12 |
| *** Guest77301 is now known as dims_ | 22:12 | |
| mtreinish | fungi: sure, but it would be nice to have a master switch or big red button in the completely fubar ci case | 22:12 |
| *** ayoung has joined #openstack-infra | 22:13 | |
| *** sdake has quit IRC | 22:14 | |
| fungi | mtreinish: project-wide big red switches are on the infra team, so ideally we'd keep their use to a minimum | 22:14 |
| *** jcoufal has quit IRC | 22:14 | |
| *** eharney has quit IRC | 22:15 | |
| openstackgerrit | Clark Boylan proposed openstack-infra/devstack-gate: DO NOT MERGE more network overlay testing https://review.openstack.org/172227 | 22:17 |
| *** wenlock__ has joined #openstack-infra | 22:19 | |
| *** bknudson has quit IRC | 22:20 | |
| clarkb | armax: ^ if that works then maybe thats what we should do, I will rewrite that function to only do one gre bridge at a time and be more similar to the ovs_gre_bridge function since I much prefer that interface | 22:21 |
| armax | clarkb: looking | 22:21 |
| clarkb | armax: its basically not using ovs for gre | 22:22 |
| clarkb | only for br-ex | 22:22 |
| *** wenlock__ has quit IRC | 22:23 | |
| armax | clarkb: ok, incidentally we had a nasty failure on http://logs.openstack.org/09/171809/4/experimental/check-tempest-dsvm-neutron-aiopcpu-full/bb9b312/logs/ which was based on the earlier version of ovs_gre_bridge right before your most recent attempt | 22:23 |
| clarkb | armax: interesting. My hunch is that something related to test order tickles the ovs gre bridge explosion | 22:23 |
| armax | clarkb: I’d be prepared to believe that :) | 22:24 |
| armax | clarkb: your most recent change is sensible | 22:25 |
| fungi | also i find it awesome that when we start testing in new ways, we inevitable expose new bugs we didn't even know existed | 22:26 |
| armax | clarkb: I’ll give it a whirl | 22:26 |
| *** amitgandhinz has quit IRC | 22:26 | |
| *** shashankhegde has quit IRC | 22:28 | |
| clarkb | armax: my concern there is that if it is ovs and not ovs+gre having trouble then this won't actually fix it for neutron | 22:29 |
| *** rlandy has quit IRC | 22:30 | |
| armax | clarkb: true | 22:30 |
| *** rlandy has joined #openstack-infra | 22:30 | |
| openstackgerrit | Khai Do proposed openstack-infra/system-config: upgrade review-dev to Gerrit ver 2.10 https://review.openstack.org/172226 | 22:32 |
| *** emagana has joined #openstack-infra | 22:33 | |
| *** emagana_ has joined #openstack-infra | 22:35 | |
| openstackgerrit | Clark Boylan proposed openstack-infra/devstack-gate: DO NOT MERGE more network overlay testing https://review.openstack.org/172227 | 22:36 |
| *** signed8bit has quit IRC | 22:36 | |
| *** dimtruck is now known as zz_dimtruck | 22:37 | |
| *** rlandy has quit IRC | 22:37 | |
| *** bhunter71 has quit IRC | 22:38 | |
| *** emagana has quit IRC | 22:38 | |
| *** emagana_ has quit IRC | 22:39 | |
| *** shashankhegde has joined #openstack-infra | 22:43 | |
| *** yamamoto has joined #openstack-infra | 22:44 | |
| *** Swami has joined #openstack-infra | 22:45 | |
| *** emagana has joined #openstack-infra | 22:45 | |
| krotscheck | ARGH. A Kingdom for test data that reproduces a bug. | 22:46 |
| *** baoli has quit IRC | 22:46 | |
| krotscheck | (It might be a small kingdom) | 22:46 |
| *** sdake_ has quit IRC | 22:47 | |
| cinerama | it's only a model :) | 22:48 |
| clarkb | cinerama: does it come with trains? | 22:48 |
| mriedem | clarkb: in jenkins-log-client.yaml i see job filters like job-filter: '.*grenade.*' | 22:49 |
| mriedem | clarkb: i need to add indexing for the cells logs | 22:49 |
| mriedem | should i also be filtering those on job? | 22:49 |
| *** yamamoto has quit IRC | 22:49 | |
| cinerama | https://s-media-cache-ak0.pinimg.com/236x/5a/b2/3c/5ab23c12b88efef953d269aff3474b7b.jpg | 22:49 |
| clarkb | mriedem: that filter says only index these logs for a grenade job | 22:50 |
| clarkb | mriedem: since cells is for not grenade you don't have to worry about that | 22:50 |
| mriedem | ok, and since cells logs only show up in cells jobs, makes sense to filter on cells | 22:50 |
| mriedem | yeah, was using grenade as a reference | 22:50 |
| *** gyee has quit IRC | 22:51 | |
| *** dims_ has quit IRC | 22:52 | |
| *** bswartz has joined #openstack-infra | 22:52 | |
| openstackgerrit | Matt Riedemann proposed openstack-infra/system-config: Index cells logs https://review.openstack.org/172231 | 22:52 |
| mriedem | blam ^ | 22:52 |
| *** sarob has joined #openstack-infra | 22:54 | |
| krotscheck | Can I get some extra eyes on a problem? I'm trying to figure out why this query returns stories that aren't actually in the filtering group: https://storyboard.openstack.org/api/v1/stories?project_group_id=57&sort_dir=desc&sort_field=id&status=active&marker=2000235&limit=10 | 22:54 |
| *** gordc has quit IRC | 22:55 | |
| krotscheck | It's got to be somewhere in http://git.openstack.org/cgit/openstack-infra/storyboard/tree/storyboard/db/api/stories.py#n125 | 22:56 |
| jhesketh | Morning | 22:57 |
| *** emagana has quit IRC | 22:58 | |
| *** sarob has quit IRC | 22:58 | |
| krotscheck | Oh. | 22:58 |
| *** ayoung has quit IRC | 22:59 | |
| clarkb | armax: just poking around with the linux bridge gre thing I am not convinced it will work. I cannot ping the 172.24.4.[12] addrs from the other host | 23:02 |
| clarkb | armax: but maybe thats ok, will have to see how tempest does | 23:02 |
| clarkb | armax: I can note that it is also an arp problem but according to tcpdump the arp who has requests make it across the gre tunnel | 23:02 |
| armax | clarkb: that’s still n-net right? | 23:02 |
| *** claudiub has quit IRC | 23:02 | |
| clarkb | armax: no neutron has it on the pub_if that is attached ot br-ed | 23:03 |
| * krotscheck facepalms | 23:03 | |
| clarkb | jhesketh: good morning | 23:03 |
| clarkb | armax: https://jenkins07.openstack.org/job/check-tempest-dsvm-neutron-aiopcpu-full/54/console that job, we will just have to see how tempest does | 23:03 |
| clarkb | jhesketh: we should make a decision on how to move https://review.openstack.org/#/c/107267/ forward | 23:04 |
| pleia2 | cinerama: poking through trusty console log to see what made puppet unhappy here | 23:04 |
| cinerama | pleia2: thanks | 23:04 |
| *** mattfarina has quit IRC | 23:05 | |
| cinerama | pleia2: got it, i think | 23:06 |
| *** btully has quit IRC | 23:06 | |
| pleia2 | cinerama: ok, great | 23:06 |
| openstackgerrit | Stephanie Miller proposed openstack-infra/system-config: Add infra setup for zanata https://review.openstack.org/169074 | 23:08 |
| *** mriedem is now known as mriedem_away | 23:11 | |
| *** dangers is now known as dangers_away | 23:11 | |
| pleia2 | cinerama: I think you can drop db_username entirely, it's the same as the default in our module | 23:12 |
| cinerama | pleia2: wasn't sure if we wanted to leave that configurable or not | 23:13 |
| pleia2 | we can always add it later | 23:13 |
| *** r-daneel has quit IRC | 23:13 | |
| cinerama | pleia2: but if you think not, we'll drop it in translate_dev.pp | 23:13 |
| openstackgerrit | Michael Krotscheck proposed openstack-infra/storyboard: Filtering by project group now works. https://review.openstack.org/172235 | 23:13 |
| krotscheck | 1 line producting bug on StoryBoard. Anyone want to take a quick look? ^^ | 23:14 |
| krotscheck | Urm, 3 line. | 23:14 |
| krotscheck | :D | 23:14 |
| pleia2 | cinerama: doh, didn't update copyright, *next next* time :) | 23:15 |
| openstackgerrit | Stephanie Miller proposed openstack-infra/system-config: Add infra setup for zanata https://review.openstack.org/169074 | 23:16 |
| pleia2 | ah, there we go | 23:16 |
| *** gyee has joined #openstack-infra | 23:16 | |
| *** ddieterl_ has quit IRC | 23:19 | |
| *** Krinkle|detached is now known as Krinkle | 23:20 | |
| openstackgerrit | Victor Stinner proposed openstack-infra/jenkins-job-builder: Fix Builder.update_job() on Python 3 https://review.openstack.org/172238 | 23:23 |
| pleia2 | cinerama: ah, we need https://review.openstack.org/#/c/171791/ in before it'll pass | 23:26 |
| pleia2 | re: wildfly::install | 23:26 |
| *** wenlock has quit IRC | 23:26 | |
| jhesketh | clarkb: yeah we should. I went to look at it yesterday but didn't get anywhere. I think the generators being classes makes a lot of sense. We also need to improve what view to use and make some of that logic clearer and maybe configurable | 23:27 |
| cinerama | pleia2: yeah we already merged the wildfly upgrade so we need to have the other wildfly change in for stuff to work | 23:27 |
| *** wenlock_ has quit IRC | 23:27 | |
| * pleia2 nods | 23:27 | |
| *** mattfarina has joined #openstack-infra | 23:27 | |
| clarkb | jhesketh: do you think the different target paths eg htmlify/ and passthrough/ make sense? | 23:28 |
| cinerama | pleia2: still i'd far rather we have that in now than start off with a stale version & make a big version jump when this is more prod-ready | 23:28 |
| pleia2 | cinerama: well, it won't pass the gate tests, so the other one actually does need to land before this one can merge | 23:28 |
| *** dboik has joined #openstack-infra | 23:29 | |
| cinerama | yup | 23:29 |
| jhesketh | clarkb: maybe.. os-loganalyze doesn't see that part at the moment. We could do it as a get variable | 23:29 |
| pleia2 | and I agree re: fresh to begin with | 23:29 |
| *** davideagnello has quit IRC | 23:29 | |
| clarkb | jhesketh: it sees in the env context passed in I think | 23:29 |
| *** dboik_ has joined #openstack-infra | 23:29 | |
| clarkb | jhesketh: wsgi gives it the full path and there is currently a check to make sure it was called with htmlify/ | 23:30 |
| openstackgerrit | Victor Stinner proposed openstack-infra/project-config: Fix Python 3 issues https://review.openstack.org/172239 | 23:30 |
| jhesketh | clarkb: oh okay. Makes sense then | 23:31 |
| krotscheck | pleia2: Hey, could you look at something for me? https://review.openstack.org/#/c/172235/ | 23:33 |
| *** hichihara has joined #openstack-infra | 23:33 | |
| *** dboik has quit IRC | 23:33 | |
| openstackgerrit | Khai Do proposed openstack-infra/system-config: upgrade review-dev to Gerrit ver 2.10 https://review.openstack.org/172226 | 23:33 |
| openstackgerrit | Timo Tijhof proposed openstack-infra/zuul: Status: Don't raise "abort" as error to the user https://review.openstack.org/172241 | 23:34 |
| *** jcoufal has joined #openstack-infra | 23:34 | |
| pleia2 | krotscheck: done | 23:36 |
| krotscheck | pleia2: Thanks :) | 23:36 |
| krotscheck | (Told you I'd be pestering you :D ) | 23:37 |
| *** davideagnello has joined #openstack-infra | 23:37 | |
| pleia2 | hehe | 23:37 |
| *** tnovacik has quit IRC | 23:37 | |
| *** crc32 has quit IRC | 23:38 | |
| *** dims_ has joined #openstack-infra | 23:38 | |
| *** sabeen3 has quit IRC | 23:38 | |
| clarkb | jhesketh: so we should merge 107267, then plan to follow up wiht both a new class for for the get generator stuff and better configuration of when to use passthrough based on the path prefix? | 23:40 |
| clarkb | armax: tempest passed with my ovs hack atop the old gre bridge | 23:41 |
| armax | clarkb: woot? | 23:41 |
| *** mattfarina has quit IRC | 23:41 | |
| clarkb | armax: sorry it passed with neutron but not nova net | 23:41 |
| clarkb | armax: the weird thing there is nova net has been passing with that setup so unsure of why that is. I am running tests again to see if it is consistent | 23:42 |
| armax | clarkb: but it did so even when we were not setting the ip’s on the interfaces | 23:42 |
| armax | clarkb: I am wondering if that shelves test is broken regardless | 23:42 |
| clarkb | armax: ya, I think that is because it is enough to have the ip set by devstack on the controllers interface | 23:42 |
| armax | clarkb: do we have a known baseline? | 23:42 |
| openstackgerrit | Merged openstack-infra/storyboard: Filtering by project group now works. https://review.openstack.org/172235 | 23:42 |
| clarkb | armax: that is enough to set up tempests route to VMs properly | 23:42 |
| clarkb | armax: yes both nova net and neutron without dvr pass currently (or did as of yesterday | 23:43 |
| *** ajmiller has quit IRC | 23:43 | |
| clarkb | it may have been a fluke, normal errors related to clouds :) | 23:43 |
| clarkb | so rerunning to see if it is consistent | 23:44 |
| *** Sukhdev has quit IRC | 23:45 | |
| *** bknudson has joined #openstack-infra | 23:45 | |
| cinerama | could we get a review/A on https://review.openstack.org/#/c/171791/ so we can land the other bits once they pass tests? | 23:45 |
| clarkb | cinerama: oh hrm my +2 didn't stick after your rebase? | 23:46 |
| clarkb | ah there was a change | 23:46 |
| cinerama | clarkb: i made one tiny change after your +2 to fix a dependency issue that i hadn't seen before | 23:46 |
| armax | clarkb: btw | 23:46 |
| armax | clarkb: If I look here: | 23:46 |
| armax | https://review.openstack.org/#/c/172227/2/functions.sh | 23:46 |
| armax | htose instructions are done by devstack too | 23:46 |
| clarkb | cinerama: approved | 23:47 |
| armax | clarkb: so I thiink we could take them out of d-g | 23:47 |
| cinerama | clarkb: thank you very much | 23:47 |
| armax | clarkb: well | 23:47 |
| clarkb | armax: devstack does not add a port to br-ex | 23:47 |
| armax | clarkb: we’d need the port | 23:47 |
| clarkb | armax: which we need to set up the l2 overlay | 23:47 |
| armax | clarkb: that’s right | 23:47 |
| armax | clarkb: I wonder if we could put that in devstack instead | 23:47 |
| clarkb | armax: ya sdague said that the new neutron lib would likely deal wiht that | 23:48 |
| clarkb | so hopefully we can simplify this in the future | 23:48 |
| armax | clarkb: that way d-g wouldn’t be touched minimally | 23:48 |
| armax | clarkb: I can whip something up in devstack to try that | 23:48 |
| armax | clarkb: assumed you think this is the way we should proceed on | 23:48 |
| clarkb | armax: lets see if the nova net tests are happy | 23:49 |
| armax | clarkb: I think that’s probably the sanest thing to do | 23:49 |
| clarkb | armax: but I am begining to think this is our path forward | 23:49 |
| armax | clarkb: because nova becomes largely unaffected | 23:49 |
| clarkb | ya | 23:49 |
| armax | clarkb: and we’ll tell devstack to circuilt br-ex to br-pub | 23:49 |
| armax | clarkb: I am pretty sure there’s already something like this in devstack | 23:50 |
| clarkb | armax: there is but only for provider networks I think | 23:50 |
| pleia2 | cinerama: once that one merges, just run recheck on your other change so we can make sure that's the last of the niggly issues | 23:50 |
| armax | clarkb: right…so it | 23:50 |
| clarkb | armax: I spent a bit of time trying to figure out what devstack wanted there before realizing it expects us to set up br-ex with ports ahead of time | 23:50 |
| armax | clarkb: good lad! | 23:50 |
| clarkb | armax: however, I would be quite happy to be told I am wrong :) | 23:51 |
| armax | clarkb: I’d need to double check | 23:51 |
| armax | clarkb: going through bash is not my forte | 23:51 |
| armax | clarkb: anyhow, let’s see what this last run tells us, I guess and I can double check that this is indeed the sanest environment for neutron dvr | 23:52 |
| jhesketh | clarkb: yep, that sounds good to me (re os-loganalyze plan) | 23:52 |
| *** Krinkle is now known as Krinkle|detached | 23:52 | |
| clarkb | armax: sounds good, if it is happy I can clean up gre_bridge to be less insane | 23:52 |
| armax | clarkb: I think we’re getting close, hopefully close enough that we can have the end in sight | 23:52 |
| clarkb | armax: ++ | 23:53 |
| armax | clarkb: ok, ping me anytime and I’ll fold the changes in the parent patch and tweak Devstack to close the circle | 23:53 |
| Swami | armax: clarkb: are we now falling back to gre_bridge instead of the ovs_gre_bridge | 23:53 |
| clarkb | Swami: yes, ovs gre bridges seem unreliable for neutrpm and nova | 23:53 |
| *** SumitNaiksatam has quit IRC | 23:53 | |
| clarkb | Swami: they drop arp packets | 23:53 |
| armax | Swami: it looks like this is viable | 23:54 |
| armax | Swami: https://review.openstack.org/#/c/172227/2/devstack-vm-gate.sh | 23:54 |
| armax | Swami: I mean https://review.openstack.org/#/c/172227/2/functions.sh | 23:54 |
| Swami | armax: yes I had a look at the patch. | 23:54 |
| clarkb | Swami: if you can figure that out I am happy to continue trying ovs | 23:54 |
| armax | clarkb: I am sure with some dedication and access to the env we could | 23:54 |
| clarkb | but debuggimg ovs is not something I am super familiar with and when a switch drops arp packets without telling me why I get a new one | 23:54 |
| armax | clarkb: but I really see no reason why we should if we can keep the pre-existing setup to be working well with neutron | 23:55 |
| Swami | if the pre-existing one works with just adding the port then I am fine with that approach. | 23:55 |
| cinerama | pleia2: am i rechecking properly? | 23:56 |
| clarkb | armax: ya thats definitely the benefit here | 23:56 |
| openstackgerrit | Merged openstack-infra/puppet-zanata: Upgrade wildfly puppet module https://review.openstack.org/171791 | 23:56 |
| clarkb | at some point I would like to debug that ovs thing further but priority is making dvr testable in the gate on multiple nodes, not digging into ovs :) | 23:56 |
| pleia2 | cinerama: yeah, it's just recheck (but recheck no bug doesn't actually hurt) | 23:58 |
| *** ayoung has joined #openstack-infra | 23:58 | |
| *** otter768 has joined #openstack-infra | 23:59 | |
| pleia2 | cinerama: you can search for 169074 on http://status.openstack.org/zuul/ to see how it's doing | 23:59 |
| pleia2 | passed trusty \o/ | 23:59 |
| clarkb | pleia2: cinerama ok now that I have overlay network tests running again what should I do to check that zanata client install went well? | 23:59 |
| pleia2 | clarkb: oh, good question, StevenK will know | 23:59 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!
