| *** bhunter71 has joined #openstack-infra | 00:01 | |
| *** hodos has quit IRC | 00:03 | |
| *** davideagnello has quit IRC | 00:05 | |
| *** ivar-lazzaro has joined #openstack-infra | 00:06 | |
| Shrews | reed: testing available_floating_ip() and it seems to work correctly. i'm limited to only 1 floating IP on dreamhost, but i can confirm it didn't attempt to create a second floating IP when one was available | 00:07 |
|---|---|---|
| openstackgerrit | Diana Whitten proposed openstack-infra/project-config: Create a repository for stackforge/xstatic-mdi https://review.openstack.org/206754 | 00:08 |
| *** abregman has quit IRC | 00:09 | |
| openstackgerrit | Diana Whitten proposed openstack-infra/project-config: Create a repository for stackforge/xstatic-bootswatch https://review.openstack.org/206732 | 00:10 |
| *** jasondotstar has quit IRC | 00:11 | |
| reed | thanks Shrews | 00:15 |
| *** achanda has quit IRC | 00:15 | |
| *** jasondotstar has joined #openstack-infra | 00:17 | |
| *** sputnik13 has quit IRC | 00:19 | |
| *** isq_ has joined #openstack-infra | 00:20 | |
| *** isq has quit IRC | 00:20 | |
| openstackgerrit | Diana Whitten proposed openstack-infra/project-config: Create a repository for stackforge/xstatic-roboto-fontface https://review.openstack.org/206767 | 00:21 |
| openstackgerrit | Khai Do proposed openstack-infra/jenkins-job-builder: Document that jjb supports json job definitions https://review.openstack.org/206311 | 00:22 |
| openstackgerrit | Diana Whitten proposed openstack-infra/project-config: Create a repository for stackforge/xstatic-roboto-fontface https://review.openstack.org/206767 | 00:22 |
| *** esker has quit IRC | 00:23 | |
| *** david-lyle has quit IRC | 00:24 | |
| openstackgerrit | Vipul Sabhaya proposed openstack-infra/project-config: Move cue and cueclient docs to docs.o.o https://review.openstack.org/201919 | 00:25 |
| *** tonytan4ever has joined #openstack-infra | 00:30 | |
| openstackgerrit | Vipul Sabhaya proposed openstack-infra/project-config: Add check-requirements to Cue https://review.openstack.org/206771 | 00:30 |
| *** puranamr has joined #openstack-infra | 00:31 | |
| *** tsekiyam_ has joined #openstack-infra | 00:32 | |
| *** xyang1 has quit IRC | 00:33 | |
| openstackgerrit | Vipul Sabhaya proposed openstack/requirements: Add cue and python-cueclient to projects.txt https://review.openstack.org/158013 | 00:33 |
| *** changbl has joined #openstack-infra | 00:34 | |
| *** mtanino has quit IRC | 00:34 | |
| *** tsekiyama has quit IRC | 00:35 | |
| *** tsekiyam_ has quit IRC | 00:36 | |
| *** otter768 has joined #openstack-infra | 00:37 | |
| harlowja | yo yo clarkb ianw , any chance i can get eyes on https://review.openstack.org/#/c/205633/ so that i can iterate on that | 00:39 |
| *** mwagner_lap has joined #openstack-infra | 00:40 | |
| *** MarkAtwood has joined #openstack-infra | 00:40 | |
| *** Guest64977 has quit IRC | 00:42 | |
| *** MarkAtwood has quit IRC | 00:43 | |
| *** puranamr has quit IRC | 00:43 | |
| *** isviridov_away is now known as isviridov | 00:43 | |
| *** FallenPegasus has joined #openstack-infra | 00:44 | |
| vipul | how do i get a project added to this launchpad? https://launchpad.net/openstack | 00:44 |
| *** tonytan4ever has quit IRC | 00:48 | |
| *** sigmavirus24_awa is now known as sigmavirus24 | 00:48 | |
| ianw | harlowja: yeah, i need to remember the "-dsvm" matching bit | 00:50 |
| harlowja | magic -dsvm | 00:50 |
| harlowja | :) | 00:50 |
| ianw | harlowja: does it work? how are you testing locally? | 00:51 |
| harlowja | i don't have a full zuul pipeline, no, but yes, those do run locally in RHEL and ubuntu | 00:51 |
| harlowja | or thats the goal | 00:52 |
| ianw | harlowja: yeah, have you run all the puppet single_use_slave setup, etc on the node first? | 00:54 |
| harlowja | nope, can't say i have | 00:54 |
| *** bknudson has joined #openstack-infra | 00:54 | |
| *** pvaneck has quit IRC | 00:55 | |
| ianw | harlowja: yeah, so to help you iterating probably best to use project-config/tools/build-image.sh (DISTRO=centos7) and try things out using that image | 00:57 |
| harlowja | hmmmm, sure, could try that | 00:57 |
| *** ivar-laz_ has joined #openstack-infra | 00:58 | |
| ianw | might just shake out issues before you have to find them in the gate | 00:58 |
| ianw | seeing as last time i looked at the zuul gate queue, my finger got sore from all the scrolling :) | 00:59 |
| harlowja | cool, ya, i've just been running the scripts locally, not the full other stuff | 00:59 |
| harlowja | :-P | 00:59 |
| *** esmute__ has quit IRC | 00:59 | |
| *** Sherif has joined #openstack-infra | 00:59 | |
| *** dchen has joined #openstack-infra | 01:00 | |
| harlowja | may try that full other stuff soon though | 01:00 |
| *** ybathia has quit IRC | 01:00 | |
| ianw | yeah, even setting things up as close to upstream as possible, i still always manage to find differences when things actually run in the gate | 01:00 |
| harlowja | ya | 01:00 |
| harlowja | sounds like what i'm going through with that script(s) | 01:00 |
| harlowja | :-P | 01:00 |
| harlowja | this time is gonna be the 'one' though, lol | 01:01 |
| *** ivar-lazzaro has quit IRC | 01:01 | |
| harlowja | i just know it | 01:01 |
| *** ivar-laz_ has quit IRC | 01:02 | |
| *** thorst has joined #openstack-infra | 01:03 | |
| *** ashleighfarnham has quit IRC | 01:04 | |
| *** esmute__ has joined #openstack-infra | 01:04 | |
| openstackgerrit | Spencer Krum proposed openstack-infra/system-config: Make the apply-test public-hiera aware https://review.openstack.org/206779 | 01:05 |
| openstackgerrit | Spencer Krum proposed openstack-infra/system-config: Use In-Tree Public Hiera data https://review.openstack.org/206692 | 01:07 |
| *** laughterwym has joined #openstack-infra | 01:07 | |
| *** FallenPegasus has quit IRC | 01:09 | |
| *** FallenPegasus has joined #openstack-infra | 01:11 | |
| *** jasondotstar has quit IRC | 01:12 | |
| openstackgerrit | Spencer Krum proposed openstack-infra/puppet-accessbot: Basic acceptance tests https://review.openstack.org/206438 | 01:12 |
| *** thorst has quit IRC | 01:14 | |
| *** erikwilson_ has quit IRC | 01:15 | |
| *** armax has joined #openstack-infra | 01:15 | |
| *** erikwilson_ has joined #openstack-infra | 01:16 | |
| *** dimtruck is now known as zz_dimtruck | 01:16 | |
| openstackgerrit | Spencer Krum proposed openstack-infra/puppet-pgsql_backup: Add .gitreview and module boilerplate https://review.openstack.org/206362 | 01:17 |
| openstackgerrit | Spencer Krum proposed openstack-infra/puppet-pgsql_backup: Boilerplate beaker-rspec files https://review.openstack.org/206403 | 01:17 |
| *** jasondotstar has joined #openstack-infra | 01:17 | |
| *** r-daneel has quit IRC | 01:19 | |
| *** FallenPegasus has quit IRC | 01:19 | |
| *** erikwilson_ has quit IRC | 01:20 | |
| *** salv-orlando has joined #openstack-infra | 01:20 | |
| *** FallenPegasus has joined #openstack-infra | 01:21 | |
| *** jasondotstar has quit IRC | 01:22 | |
| *** salv-orlando has quit IRC | 01:25 | |
| *** patrickeast has quit IRC | 01:25 | |
| *** mriedem has quit IRC | 01:26 | |
| *** EricGonczer_ has quit IRC | 01:27 | |
| *** weshay has quit IRC | 01:42 | |
| *** esmute__ has quit IRC | 01:43 | |
| *** AzherKhan has quit IRC | 01:45 | |
| *** esmute__ has joined #openstack-infra | 01:51 | |
| *** yamamoto has joined #openstack-infra | 01:51 | |
| *** Administrator has joined #openstack-infra | 01:56 | |
| *** Administrator is now known as Guest87129 | 01:56 | |
| *** banix has joined #openstack-infra | 01:56 | |
| *** dchen has quit IRC | 01:59 | |
| *** pal has joined #openstack-infra | 02:02 | |
| *** skylerberg has quit IRC | 02:02 | |
| *** Sherif has quit IRC | 02:04 | |
| *** dingyichen has quit IRC | 02:06 | |
| *** dingyichen has joined #openstack-infra | 02:06 | |
| *** crc32 has joined #openstack-infra | 02:16 | |
| *** crc32 has quit IRC | 02:16 | |
| *** crc32 has joined #openstack-infra | 02:16 | |
| *** david-lyle has joined #openstack-infra | 02:19 | |
| *** esmute__ has quit IRC | 02:21 | |
| *** unicell has quit IRC | 02:21 | |
| *** david-lyle has quit IRC | 02:23 | |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul: DNM: attempt to debug problem with re-enqueued changes https://review.openstack.org/206795 | 02:30 |
| jeblair | jhesketh, clarkb: ^ when zuul catches up, i'd like to restart it with that manually applied | 02:30 |
| jeblair | meanwhile, i'll search previous debug logs to see if i can correlate something about this instance with the previous one | 02:31 |
| *** ajmiller_ has joined #openstack-infra | 02:32 | |
| *** baoli has joined #openstack-infra | 02:35 | |
| *** pal has quit IRC | 02:35 | |
| *** achanda has joined #openstack-infra | 02:37 | |
| openstackgerrit | Timo Tijhof proposed openstack-infra/zuul: Status: Use vertical-align for progress bar instead of margin https://review.openstack.org/170058 | 02:38 |
| openstackgerrit | Timo Tijhof proposed openstack-infra/zuul: Status: Increase width of change-progress-row-left https://review.openstack.org/170059 | 02:38 |
| *** boris-42 has quit IRC | 02:40 | |
| *** armax has quit IRC | 02:41 | |
| jhesketh | jeblair: sounds good | 02:43 |
| clarkb | wfm | 02:44 |
| *** esmute__ has joined #openstack-infra | 02:45 | |
| *** banix has quit IRC | 02:48 | |
| *** oomichi has joined #openstack-infra | 02:51 | |
| *** esmute__ has quit IRC | 02:52 | |
| *** esmute__ has joined #openstack-infra | 02:53 | |
| jeblair | jhesketh, clarkb: the previous instance does not have the same "no jobs" issue that this one has, _however_ i believe that's because it was running a zuul without Ifa43b6a2cd07d81b26dd2fe48e780c5ac94dbbb9. instead, it has a second traceback related to that which ends with: | 03:09 |
| jeblair | File "/usr/local/lib/python2.7/dist-packages/zuul/model.py", line 225, in setResult | 03:09 |
| jeblair | # Get a JobTree from a Job so we can find only its dependent jobs | 03:09 |
| jeblair | AttributeError: 'NoneType' object has no attribute 'getJobTreeForJob' | 03:09 |
| jeblair | jhesketh, clarkb: which i think is a second manifestation of the same problem | 03:10 |
| jeblair | (because if getJobTree returns None, getJobs will return [] which will quietely activate the "No jobs" condition added in Ifa43b6a2cd07d81b26dd2fe48e780c5ac94dbbb9 which we saw in yesterdays incident) | 03:11 |
| jeblair | at any rate, i don't think that tells us how things got screwed up, however, it does narrow down the list of actual bits of data that are messed up from 3 to 1, so i can focus on that one. | 03:13 |
| *** erikwilson_ has joined #openstack-infra | 03:16 | |
| *** esmute__ has quit IRC | 03:19 | |
| *** baoli has quit IRC | 03:20 | |
| *** kun_huang has joined #openstack-infra | 03:20 | |
| kun_huang | what happens on this patch https://review.openstack.org/206625 | 03:21 |
| *** erikwilson_ has quit IRC | 03:21 | |
| kun_huang | the jenkins jobs for it have run 10 hours... | 03:21 |
| *** yamahata has joined #openstack-infra | 03:22 | |
| jeblair | kun_huang: check http://status.openstack.org/zuul/ | 03:22 |
| kun_huang | checked https://jenkins07.openstack.org/job/gate-tempest-dsvm-neutron-linuxbridge/201/console is running | 03:23 |
| *** alexsyip has joined #openstack-infra | 03:25 | |
| jeblair | jhesketh, clarkb: i have a very strong suspicion that the gerrit thread could cause this if it updates information for a change during the reconfiguration event (it runs outside the main thread) | 03:25 |
| jeblair | jhesketh, clarkb: however, there should be log evidence of that happening yesterday, and i don't see it | 03:26 |
| jeblair | "INFO zuul.Gerrit: Updating information for 200921,15" only happens once, and not during a reconfig | 03:26 |
| *** coolsvap|away is now known as coolsvap | 03:26 | |
| kun_huang | jeblair: that job isn't blocked, but where other 10 hours went? | 03:27 |
| *** bhunter71 has quit IRC | 03:28 | |
| *** isviridov is now known as isviridov_away | 03:28 | |
| jeblair | jhesketh, clarkb: oh drat, i'm wrong -- gerrit trigger should not be able to cause that because it has proper locking in place... | 03:29 |
| jeblair | kun_huang: the system is behind due to a bug that stopped all processing yesterday | 03:29 |
| jeblair | kun_huang: when the red line on the "Zuul Job Queue" graph hits zero, it will be caught up | 03:29 |
| kun_huang | jeblair: okay, thank you very much :) | 03:31 |
| jeblair | np | 03:31 |
| kun_huang | btw I saw the news rackspace and intel donate more machines for commnunity | 03:32 |
| kun_huang | could our CI system use some of those? | 03:32 |
| openstackgerrit | Merged openstack/requirements: Add networking-bgpvpn to projects.txt https://review.openstack.org/198411 | 03:33 |
| openstackgerrit | Merged openstack/requirements: Use oslotest 1.9.0 https://review.openstack.org/201308 | 03:33 |
| SpamapS | oh wow.. cloud-utils 'growpart' is really complicated. | 03:33 |
| SpamapS | greghaynes: ^ | 03:33 |
| *** david-lyle has joined #openstack-infra | 03:34 | |
| SpamapS | greghaynes: read 'growpart' from cloud-utils some time.. it seems crazy complex just to grow the root partition. | 03:34 |
| *** AzherKhan has joined #openstack-infra | 03:34 | |
| *** armax has joined #openstack-infra | 03:35 | |
| SpamapS | greghaynes: we may just want to get brute-forcey and slap sfdisk's in there manually. | 03:36 |
| *** ajmiller_ has quit IRC | 03:37 | |
| *** ayoung has quit IRC | 03:38 | |
| *** radez is now known as radez_g0n3 | 03:39 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/diskimage-builder: Updated from global requirements https://review.openstack.org/201852 | 03:39 |
| *** Guest87129 has quit IRC | 03:39 | |
| *** pal has joined #openstack-infra | 03:40 | |
| *** fawadkhaliq has joined #openstack-infra | 03:42 | |
| *** otter768 has quit IRC | 03:43 | |
| *** Administrator has joined #openstack-infra | 03:44 | |
| *** Administrator is now known as Guest79706 | 03:44 | |
| openstackgerrit | Merged openstack-infra/subunit2sql: Don't try to create indexes twice https://review.openstack.org/196193 | 03:45 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/diskimage-builder: Updated from global requirements https://review.openstack.org/201852 | 03:45 |
| *** fedexo has joined #openstack-infra | 03:46 | |
| *** alexsyip has quit IRC | 03:47 | |
| *** BharatK has joined #openstack-infra | 03:49 | |
| *** achanda has quit IRC | 03:55 | |
| *** amotoki has joined #openstack-infra | 03:56 | |
| nibalizer | pabelanger: ah dangit we actually did the whole 'split the apply test results' which makes it afully hard to find the error | 03:56 |
| openstackgerrit | Merged openstack/requirements: List 'singledispatch' in global-requirements https://review.openstack.org/204199 | 03:59 |
| openstackgerrit | Spencer Krum proposed openstack-infra/system-config: Use In-Tree Public Hiera data https://review.openstack.org/206692 | 04:00 |
| openstackgerrit | Spencer Krum proposed openstack-infra/system-config: Make the apply-test public-hiera aware https://review.openstack.org/206779 | 04:00 |
| *** devkulkarni has joined #openstack-infra | 04:01 | |
| *** jhesketh has quit IRC | 04:01 | |
| *** jhesketh has joined #openstack-infra | 04:02 | |
| *** isviridov_away is now known as isviridov | 04:04 | |
| openstackgerrit | Merged openstack/requirements: Move integration test dependencies into bindep https://review.openstack.org/203358 | 04:07 |
| openstackgerrit | Spencer Krum proposed openstack-infra/puppet-pgsql_backup: Add .gitreview and module boilerplate https://review.openstack.org/206362 | 04:10 |
| openstackgerrit | Spencer Krum proposed openstack-infra/puppet-pgsql_backup: Boilerplate beaker-rspec files https://review.openstack.org/206403 | 04:10 |
| openstackgerrit | Merged openstack/requirements: Document current constraints status. https://review.openstack.org/201893 | 04:12 |
| openstackgerrit | Ian Wienand proposed openstack-infra/project-config: Create gate-infra-puppet-apply-dsvm-{ostype} https://review.openstack.org/205668 | 04:12 |
| *** sigmavirus24 is now known as sigmavirus24_awa | 04:14 | |
| *** harlowja_at_home has joined #openstack-infra | 04:17 | |
| openstackgerrit | Merged openstack-infra/devstack-gate: Add keystoneauth to PROJECTS list https://review.openstack.org/206753 | 04:20 |
| *** fedexo has quit IRC | 04:21 | |
| *** isviridov is now known as isviridov_away | 04:22 | |
| *** FallenPegasus has quit IRC | 04:24 | |
| *** Sherif has joined #openstack-infra | 04:28 | |
| *** esmute__ has joined #openstack-infra | 04:33 | |
| *** esmute__ has quit IRC | 04:36 | |
| *** puranamr has joined #openstack-infra | 04:38 | |
| *** esmute__ has joined #openstack-infra | 04:41 | |
| *** armax has quit IRC | 04:42 | |
| *** yfried|afk is now known as yfried | 04:45 | |
| *** devkulkarni has quit IRC | 04:47 | |
| *** fedexo has joined #openstack-infra | 04:48 | |
| *** unicell has joined #openstack-infra | 04:49 | |
| *** davideagnello has joined #openstack-infra | 04:51 | |
| *** Guest79706 has quit IRC | 04:56 | |
| *** yfried is now known as yfried|afk | 04:57 | |
| *** Administrator has joined #openstack-infra | 04:58 | |
| *** Administrator is now known as Guest96878 | 04:58 | |
| *** esmute__ has quit IRC | 05:01 | |
| *** harlowja_still_a has joined #openstack-infra | 05:02 | |
| *** yfried|afk is now known as yfried | 05:03 | |
| *** harlowja_at_home has quit IRC | 05:05 | |
| openstackgerrit | Merged openstack-infra/project-config: Add check-requirements to Cue https://review.openstack.org/206771 | 05:06 |
| *** BharatK has quit IRC | 05:09 | |
| *** yfried has quit IRC | 05:09 | |
| *** davideagnello has quit IRC | 05:12 | |
| openstackgerrit | Merged openstack-infra/project-config: Add puppet gems for gate-puppet-openstack-integration-dsvm https://review.openstack.org/206269 | 05:13 |
| openstackgerrit | Stanislaw Pitucha proposed openstack/requirements: Minimal setuptools required for tests https://review.openstack.org/199788 | 05:13 |
| *** pahuang__ has quit IRC | 05:14 | |
| *** sputnik13 has joined #openstack-infra | 05:15 | |
| *** harlowja_still_a has quit IRC | 05:16 | |
| nibalizer | fungi: ianw I have pokethed the puppet folk https://tickets.puppetlabs.com/browse/PUP-3829 | 05:16 |
| *** harlowja_at_home has joined #openstack-infra | 05:16 | |
| *** esmute__ has joined #openstack-infra | 05:17 | |
| *** erikwilson_ has joined #openstack-infra | 05:17 | |
| *** erikwilson_ has quit IRC | 05:22 | |
| ianw | nibalizer: thanks | 05:22 |
| ianw | nibalizer: did this just change? was there a new EPEL release? | 05:23 |
| nibalizer | I'm not sure | 05:24 |
| nibalizer | I think you're in a better position to slueth that out than I am | 05:24 |
| * Clint squints. | 05:24 | |
| nibalizer | its complex for us becuase we use get-pip so we're trying to use that and pretend we are using the package | 05:24 |
| nibalizer | Clint: please do, I'm not sure I found the right issue | 05:25 |
| Clint | greghaynes: cloud-utils 0.27-2 might solve your problem | 05:25 |
| *** BharatK has joined #openstack-infra | 05:25 | |
| *** achanda has joined #openstack-infra | 05:26 | |
| *** bdemers has joined #openstack-infra | 05:27 | |
| *** sergmelikyan has joined #openstack-infra | 05:28 | |
| *** harlowja_still_a has joined #openstack-infra | 05:29 | |
| *** deepakcs has joined #openstack-infra | 05:31 | |
| *** harlowja_at_home has quit IRC | 05:32 | |
| ianw | nibalizer: it seems like it hasn't shipped the symlinks since http://pkgs.fedoraproject.org/cgit/python-pip.git/commit/?h=el6&id=dd9e7dc94db45086cb78b324a160d8f7cb27070d | 05:32 |
| nibalizer | so, before that patch, there was a pip-python and a symlink from pip -> pip-python ? and now there is only pip and no pip-python ? | 05:35 |
| openstackgerrit | Atsushi SAKAI proposed openstack-infra/shade: Fix two typos and one readablity on shade documentation https://review.openstack.org/206871 | 05:36 |
| *** kun_huang has left #openstack-infra | 05:37 | |
| *** harlowja_still_a has quit IRC | 05:38 | |
| *** Somay has joined #openstack-infra | 05:38 | |
| *** crc32 has quit IRC | 05:39 | |
| openstackgerrit | Merged openstack/requirements: Add dnspython3 to global-requirements.txt https://review.openstack.org/196633 | 05:39 |
| *** sputnik13 has quit IRC | 05:40 | |
| *** b10n1k has quit IRC | 05:41 | |
| *** otter768 has joined #openstack-infra | 05:44 | |
| *** fedexo has quit IRC | 05:45 | |
| *** kushal has quit IRC | 05:46 | |
| *** fedexo has joined #openstack-infra | 05:46 | |
| openstackgerrit | Clint 'SpamapS' Byrum proposed openstack-infra/shade: Document create_object https://review.openstack.org/206872 | 05:48 |
| *** otter768 has quit IRC | 05:48 | |
| *** yamamoto has quit IRC | 05:50 | |
| *** puranamr has quit IRC | 05:51 | |
| openstackgerrit | Ghanshyam Mann proposed openstack-infra/project-config: Remove Nova v21 neutron job & make one for v2.0 https://review.openstack.org/163731 | 05:51 |
| *** abregman has joined #openstack-infra | 05:56 | |
| openstackgerrit | Travis Tripp proposed openstack-infra/irc-meetings: Update Searchlight Team Meeting https://review.openstack.org/206875 | 05:57 |
| *** fedexo has quit IRC | 05:57 | |
| *** _nadya_ has joined #openstack-infra | 05:59 | |
| *** scheuran has joined #openstack-infra | 05:59 | |
| *** sergmelikyan has quit IRC | 06:01 | |
| *** sergmelikyan has joined #openstack-infra | 06:03 | |
| *** sergmelikyan has quit IRC | 06:03 | |
| *** vlaza has joined #openstack-infra | 06:04 | |
| *** flepied has joined #openstack-infra | 06:05 | |
| *** kiran-r has joined #openstack-infra | 06:05 | |
| *** ParsectiX has joined #openstack-infra | 06:06 | |
| *** flepied1 has quit IRC | 06:07 | |
| *** afazekas has joined #openstack-infra | 06:09 | |
| *** sergmelikyan has joined #openstack-infra | 06:09 | |
| *** esmute__ has quit IRC | 06:10 | |
| *** AJaeger has joined #openstack-infra | 06:10 | |
| AJaeger | fungi, clarkb: https://review.openstack.org/#/c/197213/ has a -1 from the Xen CI but this -1 stays there since several patch iterations, is there a way to remove the -1? It does not get reset by patch upload apparently ;( | 06:11 |
| *** hichihara has quit IRC | 06:13 | |
| AJaeger | Sorry, I'm wrong, Xen CI is run on the change set, just fails ;( | 06:13 |
| StevenK | AJaeger: Hai! So I should remove the file section, or the post job, or both? | 06:13 |
| AJaeger | StevenK: just the files section | 06:13 |
| AJaeger | it needs to be run with every merge ;( | 06:14 |
| openstackgerrit | Merged openstack-infra/project-config: Create gate-infra-puppet-apply-dsvm-{ostype} https://review.openstack.org/205668 | 06:14 |
| StevenK | It should be fast enough, but the poor proposal slave | 06:14 |
| AJaeger | Yeah, let's give him some more todo ;) | 06:16 |
| *** flepied has quit IRC | 06:16 | |
| StevenK | WCPGW, right! | 06:16 |
| StevenK | (What Could Possibly Go Wrong) | 06:16 |
| StevenK | Feels like a Friday since I'm packing for a trip. Isn't a Friday! | 06:17 |
| *** sergmelikyan has quit IRC | 06:17 | |
| AJaeger | What are doing? | 06:18 |
| StevenK | AJaeger: I'm heading 1000km north for PyCon AU in Brisbane. | 06:18 |
| StevenK | So it isn't really a flight, more than an annoyance. :-P | 06:18 |
| openstackgerrit | yolanda.robla proposed openstack-infra/puppet-gerrit: New logrotate class in gerrit https://review.openstack.org/201312 | 06:19 |
| StevenK | AJaeger: But I'm used to packing on a Friday night or a Saturday night for a weekend flight/s to a mid-cycle or a summit | 06:19 |
| *** zeih has joined #openstack-infra | 06:19 | |
| *** sergmelikyan has joined #openstack-infra | 06:21 | |
| AJaeger | I see - enjoy PyCon AU! | 06:21 |
| openstackgerrit | Steve Kowalik proposed openstack-infra/project-config: Add a register-zanata-projects.py script and job https://review.openstack.org/198236 | 06:21 |
| *** yamamoto has joined #openstack-infra | 06:22 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack-infra/project-config: Normalize projects.yaml https://review.openstack.org/206883 | 06:23 |
| *** pahuang has joined #openstack-infra | 06:24 | |
| *** vinayyadhav has joined #openstack-infra | 06:25 | |
| openstackgerrit | Sumit Naiksatam proposed openstack-infra/project-config: Add experimental scale testing job for GBP https://review.openstack.org/206680 | 06:25 |
| *** sergmelikyan has quit IRC | 06:26 | |
| *** sergmelikyan has joined #openstack-infra | 06:27 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/requirements: Updated from generate-constraints https://review.openstack.org/206650 | 06:28 |
| openstackgerrit | Sirushti Murugesan proposed openstack-infra/project-config: Make the Heat Convergence Gate job non voting https://review.openstack.org/206888 | 06:29 |
| *** jamespage_ has joined #openstack-infra | 06:30 | |
| *** armax has joined #openstack-infra | 06:33 | |
| *** yfried has joined #openstack-infra | 06:35 | |
| *** yfried has quit IRC | 06:39 | |
| *** yfried has joined #openstack-infra | 06:39 | |
| *** yfried is now known as yfried|afk | 06:41 | |
| *** flepied has joined #openstack-infra | 06:45 | |
| *** Somay has quit IRC | 06:48 | |
| *** flepied has quit IRC | 06:49 | |
| *** Somay has joined #openstack-infra | 06:50 | |
| *** jamespage_ has quit IRC | 06:51 | |
| *** jamespage_ has joined #openstack-infra | 06:51 | |
| *** flepied has joined #openstack-infra | 06:52 | |
| openstackgerrit | Travis Tripp proposed openstack-infra/irc-meetings: Update Searchlight Team Meeting https://review.openstack.org/206875 | 06:54 |
| *** achanda has quit IRC | 06:54 | |
| *** _nadya_ has quit IRC | 06:55 | |
| *** _nadya_ has joined #openstack-infra | 06:56 | |
| *** jamespage_ has quit IRC | 06:56 | |
| *** jamespage_ has joined #openstack-infra | 06:57 | |
| *** achanda has joined #openstack-infra | 06:57 | |
| *** markus_z has joined #openstack-infra | 06:58 | |
| *** kushal has joined #openstack-infra | 06:59 | |
| *** tongli_ has quit IRC | 07:00 | |
| *** _nadya_ has quit IRC | 07:00 | |
| *** sergmelikyan has quit IRC | 07:00 | |
| *** Guest96878 has quit IRC | 07:02 | |
| *** mpavone has joined #openstack-infra | 07:03 | |
| openstackgerrit | Sirushti Murugesan proposed openstack-infra/project-config: Make the Heat grenade job voting https://review.openstack.org/201052 | 07:04 |
| *** yamahata has quit IRC | 07:05 | |
| *** otherwiseguy has quit IRC | 07:05 | |
| *** HeOS has quit IRC | 07:05 | |
| *** _nadya_ has joined #openstack-infra | 07:07 | |
| *** _nadya_ has quit IRC | 07:08 | |
| *** _nadya_ has joined #openstack-infra | 07:09 | |
| *** otherwiseguy has joined #openstack-infra | 07:10 | |
| *** salv-orlando has joined #openstack-infra | 07:12 | |
| *** niska has quit IRC | 07:12 | |
| *** davideagnello has joined #openstack-infra | 07:13 | |
| *** _nadya_ has quit IRC | 07:13 | |
| *** adam_g has quit IRC | 07:14 | |
| *** yfried|afk is now known as yfried | 07:14 | |
| *** niska has joined #openstack-infra | 07:16 | |
| *** vinayyadhav has quit IRC | 07:17 | |
| *** davideagnello has quit IRC | 07:17 | |
| *** erikwilson_ has joined #openstack-infra | 07:18 | |
| *** jamespage_ has quit IRC | 07:18 | |
| *** jamespage_ has joined #openstack-infra | 07:18 | |
| *** sshnaidm has joined #openstack-infra | 07:21 | |
| *** nfedotov has joined #openstack-infra | 07:22 | |
| *** erikwilson_ has quit IRC | 07:22 | |
| *** jamespage_ has quit IRC | 07:24 | |
| *** jamespage_ has joined #openstack-infra | 07:24 | |
| *** Somay has quit IRC | 07:25 | |
| *** armax has quit IRC | 07:25 | |
| AJaeger | jhesketh, clarkb, fungi, pleia2, SergeyLukjanov, jeblair: Is anybody of you around? Seems that Zuul is stuck again ;( Check http://status.openstack.org/zuul/ and look at the top changes as well as the activity graph, our systems is really idle but we have 137 changes in the check queue waiting | 07:25 |
| *** achanda has quit IRC | 07:27 | |
| *** kun_huang has joined #openstack-infra | 07:27 | |
| kun_huang | is gate broken again? | 07:27 |
| *** arxcruz has joined #openstack-infra | 07:27 | |
| kun_huang | http://status.openstack.org/zuul/ most of jobs are passed but the first one are queued now | 07:28 |
| tangchen | Hi, does anyone know how to use a CI log server ? | 07:30 |
| tangchen | I got a script from here: wget https://raw.githubusercontent.com/rasselin/os-ext-testing/master/puppet/install_log_server.sh | 07:30 |
| AJaeger | kun_huang: yes, looks like we have a problem... | 07:30 |
| AJaeger | tangchen: best talk with the authoer of that script then - or come back during US time, hope somebody can help you then | 07:30 |
| tangchen | And when I finished to run it, I don't quite understand how to use the log server | 07:31 |
| tangchen | OK, thanks. | 07:31 |
| *** k4n0 has joined #openstack-infra | 07:31 | |
| *** fhubik has joined #openstack-infra | 07:31 | |
| kun_huang | AJaeger: :( | 07:33 |
| *** zeih has quit IRC | 07:33 | |
| *** markus_z has quit IRC | 07:34 | |
| *** Somay has joined #openstack-infra | 07:35 | |
| *** Somay has quit IRC | 07:36 | |
| *** dtantsur|afk is now known as dtantsur | 07:37 | |
| AJaeger | ttx, can you send an alert that Zuul is stuck currently, please? | 07:38 |
| *** ihrachyshka has joined #openstack-infra | 07:38 | |
| *** jlanoux has joined #openstack-infra | 07:38 | |
| *** rdopiera has joined #openstack-infra | 07:43 | |
| rdopiera | hi guys, I have a small problem with Jenkins on this patch: https://review.openstack.org/#/c/206479/ | 07:44 |
| rdopiera | basically, pep8 claims it has failed, but when I try to see the logs, I get "file not found" | 07:44 |
| *** otter768 has joined #openstack-infra | 07:44 | |
| AJaeger | rdopiera: we sometimes have problems with uploading files to our log server ;( | 07:45 |
| rdopiera | in addition, the pep8 tests pass locally, so I wonder what might be wrong | 07:46 |
| AJaeger | rdopiera: I suggest you run "tox -e pep8" locally and if that does not help, add a "recheck" comment to the change. Note that our CI is currently stuck, so it will take quite some until you get the result. The -1 could be also because of the upload problem | 07:46 |
| AJaeger | rdopiera: since you tested locally already, add "recheck" and wait patiently - might take half a day for the result to show up due to the stuck CI ;( | 07:48 |
| rdopiera | AJaeger: I see, that's exactly what I wanted to know, thank you | 07:48 |
| *** otter768 has quit IRC | 07:49 | |
| *** ParsectiX has quit IRC | 07:50 | |
| *** jamespage_ has quit IRC | 07:50 | |
| *** markus_z has joined #openstack-infra | 07:51 | |
| *** zeih has joined #openstack-infra | 07:51 | |
| *** jamespage_ has joined #openstack-infra | 07:51 | |
| *** ParsectiX has joined #openstack-infra | 07:51 | |
| *** hashar has joined #openstack-infra | 07:52 | |
| *** crc32 has joined #openstack-infra | 07:52 | |
| *** larainema has quit IRC | 07:53 | |
| *** larainema has joined #openstack-infra | 07:54 | |
| *** woodster_ has quit IRC | 07:54 | |
| *** pal has quit IRC | 07:59 | |
| *** hashar_ has joined #openstack-infra | 08:02 | |
| *** jamespage_ has quit IRC | 08:04 | |
| *** hashar has quit IRC | 08:04 | |
| *** jistr has joined #openstack-infra | 08:08 | |
| *** zeih_ has joined #openstack-infra | 08:10 | |
| *** jamespage_ has joined #openstack-infra | 08:11 | |
| *** zeih has quit IRC | 08:13 | |
| *** jamespage_ has quit IRC | 08:13 | |
| *** zeih_ has quit IRC | 08:14 | |
| *** zeih has joined #openstack-infra | 08:15 | |
| *** dguitarbite has joined #openstack-infra | 08:18 | |
| *** jamespage_ has joined #openstack-infra | 08:18 | |
| *** ifarkas has joined #openstack-infra | 08:18 | |
| *** mpaolino has joined #openstack-infra | 08:20 | |
| *** jasondotstar has joined #openstack-infra | 08:25 | |
| *** sshnaidm has quit IRC | 08:26 | |
| *** sshnaidm has joined #openstack-infra | 08:26 | |
| *** jamespage_ has quit IRC | 08:28 | |
| *** zeih_ has joined #openstack-infra | 08:30 | |
| *** jamespage_ has joined #openstack-infra | 08:31 | |
| *** fhubik is now known as fhubik_afk | 08:31 | |
| *** zeih has quit IRC | 08:33 | |
| *** fhubik_afk is now known as fhubik | 08:38 | |
| *** e0ne has joined #openstack-infra | 08:40 | |
| *** markus_z has quit IRC | 08:43 | |
| *** jamespage_ has quit IRC | 08:43 | |
| *** e0ne has quit IRC | 08:43 | |
| *** markus_z has joined #openstack-infra | 08:43 | |
| openstackgerrit | Yanis Guenane proposed openstack-infra/project-config: msync: Create a post-merged hook https://review.openstack.org/189216 | 08:43 |
| *** Sherif has quit IRC | 08:44 | |
| *** HeOS has joined #openstack-infra | 08:47 | |
| *** _nadya_ has joined #openstack-infra | 08:47 | |
| openstackgerrit | Yanis Guenane proposed openstack-infra/project-config: msync: Create a post-merged hook https://review.openstack.org/189216 | 08:47 |
| *** Ala has joined #openstack-infra | 08:49 | |
| *** zeih_ has quit IRC | 08:52 | |
| *** dingyichen has quit IRC | 08:53 | |
| *** bexelbie has joined #openstack-infra | 08:53 | |
| *** ZZelle has quit IRC | 08:53 | |
| *** ZZelle has joined #openstack-infra | 08:53 | |
| *** pelix has joined #openstack-infra | 08:55 | |
| *** crc32 has quit IRC | 08:56 | |
| *** pal has joined #openstack-infra | 08:57 | |
| *** doude has joined #openstack-infra | 09:00 | |
| *** nijaba has quit IRC | 09:02 | |
| *** markus_z has quit IRC | 09:02 | |
| *** coolsvap is now known as coolsvap|away | 09:04 | |
| *** jtomasek has quit IRC | 09:05 | |
| *** amotoki has quit IRC | 09:05 | |
| *** BharatK has quit IRC | 09:05 | |
| *** jtomasek has joined #openstack-infra | 09:06 | |
| *** fhubik is now known as fhubik_afk | 09:06 | |
| *** nijaba has joined #openstack-infra | 09:07 | |
| *** fhubik_afk is now known as fhubik | 09:08 | |
| *** jschwarz has joined #openstack-infra | 09:10 | |
| jschwarz | hi guys... is there a problem with the gate? | 09:11 |
| kun_huang | jschwarz: yep, blocked | 09:11 |
| jschwarz | I have a patch here https://review.openstack.org/#/c/205796/ which was approved but didn't get a -1 from jenkins and it isn't on the zuul list | 09:11 |
| *** BobBall_AWOL is now known as BobBall | 09:11 | |
| *** dtantsur is now known as dtantsur|brb | 09:12 | |
| *** coolsvap|away is now known as coolsvap | 09:12 | |
| *** hashar has joined #openstack-infra | 09:13 | |
| *** ihrachyshka has quit IRC | 09:15 | |
| *** hashar_ has quit IRC | 09:15 | |
| *** rdopiera has left #openstack-infra | 09:16 | |
| *** erikwilson_ has joined #openstack-infra | 09:19 | |
| *** markus_z has joined #openstack-infra | 09:19 | |
| *** alexpilotti has joined #openstack-infra | 09:21 | |
| AJaeger | jschwarz: yes, unfortunately ;( | 09:22 |
| AJaeger | jschwarz: we're currently waiting for an infra root to wake up... | 09:22 |
| AJaeger | ttx, lifeless: Are you around? | 09:23 |
| *** erikwilson_ has quit IRC | 09:24 | |
| ttx | AJaeger: o/ | 09:24 |
| jschwarz | thanks | 09:25 |
| AJaeger | ttx, zuul is stuck, could you send out a status update, please? | 09:25 |
| AJaeger | There'S no infra root around ;( | 09:25 |
| ttx | sure, what's the message ? | 09:25 |
| kun_huang | haha ttx is herer | 09:25 |
| ttx | "zuul is stuck" ? | 09:26 |
| AJaeger | What about: "Currently our CI system is broken, jobs are not getting processed at all." | 09:26 |
| spredzy | Does someone can tell me how can I remove a remote branch on a project ? I get kicked with "You need 'Push' rights with the 'Force Push'" | 09:26 |
| ttx | #status alert Currently our CI system is broken, jobs are not getting processed at all. | 09:26 |
| openstackstatus | ttx: sending alert | 09:26 |
| AJaeger | zuul is stuck works as well ;) | 09:26 |
| *** yamamoto has quit IRC | 09:26 | |
| AJaeger | spredzy: only infra roots can remove branches - please come back during US time | 09:26 |
| ttx | AJaeger: you should really ask for the keys to send messages :) | 09:26 |
| spredzy | AJaeger, ok thank you for the answer. Will do. | 09:27 |
| AJaeger | ttx, I still hope that we could solve that differently ;) | 09:27 |
| AJaeger | ttx, thanks! | 09:27 |
| ttx | AJaeger: it's an alert, so it will overtake channel topics and all... we'l need to clear it | 09:28 |
| *** abregman is now known as abregman|brb | 09:28 | |
| -openstackstatus- NOTICE: Currently our CI system is broken, jobs are not getting processed at all. | 09:28 | |
| *** ChanServ changes topic to "Currently our CI system is broken, jobs are not getting processed at all." | 09:28 | |
| kun_huang | ttx https://review.openstack.org/#/c/206740/ can we block this one? | 09:29 |
| *** yfried is now known as yfried|afk | 09:29 | |
| AJaeger | ttx: I expect that one of the infra roots clears it once zuul is up again | 09:29 |
| AJaeger | kun_huang: what do you mean with block? | 09:29 |
| kun_huang | most of patches after this one have passed the jenkins jobs | 09:29 |
| kun_huang | only the first one is queued | 09:29 |
| AJaeger | kun_huang: wait until we have restarted zuul and then the infra roots might be able to requeue events. | 09:30 |
| kun_huang | maybe abandon the first one, and allow other patches first | 09:30 |
| kun_huang | AJaeger: okay | 09:30 |
| kun_huang | btw is that a bug? | 09:30 |
| AJaeger | kun_huang: Let's not micro-optimize ;) | 09:30 |
| openstackstatus | ttx: finished sending alert | 09:31 |
| AJaeger | ttx, I'll send an email out as well... | 09:31 |
| ttx | ack | 09:31 |
| spredzy | AJaeger, can I get the nick of one or two -infra roots so I can ping them later ? | 09:33 |
| AJaeger | spredzy: Just ask here - but please wait until the CI system is working again, that has priority. | 09:34 |
| AJaeger | spredzy: or send an email to openstack-infra mailing list... | 09:34 |
| spredzy | AJaeger, ack and sure thing. | 09:34 |
| *** yamamoto has joined #openstack-infra | 09:35 | |
| greghaynes | sad zuul :( | 09:35 |
| *** ParsectiX has quit IRC | 09:35 | |
| rcarrillocruz | :/ | 09:37 |
| rcarrillocruz | greghaynes: still in Europe? | 09:37 |
| greghaynes | Yep | 09:37 |
| greghaynes | SpamapS: yes, that is one of the big wins of making growroot not a by redauly included package - its not that hard to make an element DTRT for your specific case | 09:39 |
| greghaynes | er, default | 09:43 |
| *** vinay_yadhav has joined #openstack-infra | 09:45 | |
| *** otter768 has joined #openstack-infra | 09:45 | |
| *** markus_z has quit IRC | 09:46 | |
| *** mdbooth has quit IRC | 09:46 | |
| *** mdbooth has joined #openstack-infra | 09:47 | |
| *** alex_xu has quit IRC | 09:48 | |
| *** yfried|afk is now known as yfried | 09:49 | |
| *** alex_xu has joined #openstack-infra | 09:49 | |
| *** otter768 has quit IRC | 09:50 | |
| *** coolsvap is now known as coolsvap|away | 09:50 | |
| *** ihrachyshka has joined #openstack-infra | 09:53 | |
| *** doude has quit IRC | 09:55 | |
| *** e0ne has joined #openstack-infra | 09:57 | |
| *** jasondotstar has quit IRC | 09:57 | |
| *** coolsvap|away is now known as coolsvap | 09:58 | |
| *** yfried is now known as yfried|afk | 09:59 | |
| *** ihrachyshka has quit IRC | 10:00 | |
| *** ifarkas has quit IRC | 10:00 | |
| *** nfedotov has quit IRC | 10:00 | |
| *** rhe00 has quit IRC | 10:00 | |
| *** Kennan has quit IRC | 10:00 | |
| *** melwitt has quit IRC | 10:00 | |
| *** Swanson has quit IRC | 10:00 | |
| *** mjturek1 has quit IRC | 10:00 | |
| *** jogo has quit IRC | 10:00 | |
| *** dguerri` has quit IRC | 10:00 | |
| *** sc68cal has quit IRC | 10:00 | |
| *** melwitt has joined #openstack-infra | 10:00 | |
| *** Swanson has joined #openstack-infra | 10:01 | |
| *** ifarkas has joined #openstack-infra | 10:01 | |
| *** ihrachyshka has joined #openstack-infra | 10:01 | |
| *** Kennan has joined #openstack-infra | 10:01 | |
| *** dguerri` has joined #openstack-infra | 10:01 | |
| *** dguerri` is now known as dguerri | 10:01 | |
| *** dguerri has joined #openstack-infra | 10:01 | |
| *** mjturek1 has joined #openstack-infra | 10:01 | |
| *** nfedotov has joined #openstack-infra | 10:01 | |
| *** rhe00 has joined #openstack-infra | 10:01 | |
| *** sc68cal has joined #openstack-infra | 10:02 | |
| *** Volundr has joined #openstack-infra | 10:02 | |
| *** e0ne_ has joined #openstack-infra | 10:02 | |
| *** kiran-r has quit IRC | 10:03 | |
| *** pelix1 has joined #openstack-infra | 10:04 | |
| *** pelix has quit IRC | 10:04 | |
| *** pelix1 is now known as pelix | 10:04 | |
| *** pelix has quit IRC | 10:04 | |
| *** pelix has joined #openstack-infra | 10:04 | |
| *** Hefeweiz1n has joined #openstack-infra | 10:05 | |
| *** Ala has quit IRC | 10:05 | |
| *** AJaeger has quit IRC | 10:05 | |
| *** whoops has quit IRC | 10:05 | |
| *** Hefeweizen has quit IRC | 10:05 | |
| *** e0ne has quit IRC | 10:05 | |
| *** jschwarz has quit IRC | 10:05 | |
| *** ildikov has quit IRC | 10:05 | |
| *** kaisers has quit IRC | 10:05 | |
| *** eantyshev has quit IRC | 10:05 | |
| *** ildikov has joined #openstack-infra | 10:05 | |
| *** e0ne_ is now known as e0ne | 10:06 | |
| *** jogo has joined #openstack-infra | 10:06 | |
| *** kaisers has joined #openstack-infra | 10:06 | |
| *** jschwarz has joined #openstack-infra | 10:06 | |
| *** BharatK has joined #openstack-infra | 10:08 | |
| *** eantyshev has joined #openstack-infra | 10:08 | |
| *** jasondotstar has joined #openstack-infra | 10:09 | |
| *** fhubik is now known as fhubik_afk | 10:10 | |
| *** kiran-r has joined #openstack-infra | 10:12 | |
| *** ParsectiX has joined #openstack-infra | 10:17 | |
| *** AJaeger has joined #openstack-infra | 10:17 | |
| *** Ala has joined #openstack-infra | 10:18 | |
| *** pc_m has joined #openstack-infra | 10:18 | |
| *** doude has joined #openstack-infra | 10:18 | |
| *** vkramskikh has joined #openstack-infra | 10:21 | |
| *** _nadya_ has quit IRC | 10:23 | |
| spredzy | AJaeger, ping https://review.openstack.org/#/c/189216/. Does the explanation make sense on that review ? Do you agree with them ? | 10:24 |
| spredzy | Will update the commit msg to adapt to your comment, I'll check with our PTL to which page we should point to first. | 10:25 |
| *** dtantsur|brb is now known as dtantsur | 10:30 | |
| AJaeger | spredzy: thanks! I need more time for this review, will do later. Thanks a lot for running this manually for me, that helped! | 10:31 |
| spredzy | AJaeger, yw. Sure I'll check your comment later then. | 10:32 |
| *** soren has joined #openstack-infra | 10:36 | |
| *** abregman|brb is now known as abregman | 10:37 | |
| *** rossella_s has quit IRC | 10:37 | |
| *** rossella_s has joined #openstack-infra | 10:38 | |
| dims | looks like zuul caught up with the huge backlog yesterday | 10:38 |
| *** vlaza has left #openstack-infra | 10:40 | |
| AJaeger | dims: it did - and now it decided to take a nap ;( | 10:42 |
| AJaeger | dims, see topic | 10:42 |
| dims | awww... | 10:42 |
| dims | i got happy too soon :) | 10:43 |
| sdague | oh neat, all the clouds fell over again? | 10:45 |
| sdague | SergeyLukjanov / jhesketh either of you able to poke at that? | 10:46 |
| *** yfried|afk is now known as yfried | 10:48 | |
| *** lennyb_ has quit IRC | 10:49 | |
| *** mrmartin has joined #openstack-infra | 10:50 | |
| *** markus_z has joined #openstack-infra | 10:53 | |
| *** cdent has joined #openstack-infra | 10:55 | |
| *** coolsvap is now known as coolsvap|away | 10:55 | |
| *** yamamoto has quit IRC | 10:57 | |
| *** yamamoto has joined #openstack-infra | 10:59 | |
| *** mpaolino has quit IRC | 11:00 | |
| *** fhubik_afk is now known as fhubik | 11:00 | |
| AJaeger | sdague: nobody around so far ;( | 11:00 |
| *** zeih has joined #openstack-infra | 11:03 | |
| sdague | bummer | 11:06 |
| *** AzherKhan has quit IRC | 11:07 | |
| *** hashar has quit IRC | 11:13 | |
| *** erikwilson_ has joined #openstack-infra | 11:20 | |
| *** bexelbie has quit IRC | 11:21 | |
| *** bexelbie has joined #openstack-infra | 11:21 | |
| *** jasondot_ has joined #openstack-infra | 11:23 | |
| *** baoli has joined #openstack-infra | 11:24 | |
| *** erikwilson_ has quit IRC | 11:24 | |
| *** e0ne has quit IRC | 11:25 | |
| *** jasondotstar has quit IRC | 11:25 | |
| AJaeger | Let's hope that fungi or clarkb wake up early and can fix it. I noticed this 4 hours ago ;( | 11:29 |
| *** zeih has quit IRC | 11:29 | |
| *** juggler has quit IRC | 11:30 | |
| *** baoli has quit IRC | 11:31 | |
| *** zeih has joined #openstack-infra | 11:31 | |
| *** juggler has joined #openstack-infra | 11:33 | |
| *** mwagner_lap has quit IRC | 11:35 | |
| *** e0ne has joined #openstack-infra | 11:35 | |
| *** oomichi has quit IRC | 11:36 | |
| *** olaph has quit IRC | 11:37 | |
| *** pal has quit IRC | 11:37 | |
| jhesketh | sdague, AJaeger: hmm, looks bad | 11:39 |
| jhesketh | I'll see if I can poke around | 11:39 |
| sdague | jhesketh: yeh, it looks like another zuul crash of some sort | 11:41 |
| jhesketh | sdague: looks like the disk is full, which is different to last time | 11:41 |
| *** ldnunes has joined #openstack-infra | 11:43 | |
| *** zeih has quit IRC | 11:44 | |
| * cdent wishes he had some hardware and personal clone or 5 or 6 to devote to infra | 11:44 | |
| *** bswartz has quit IRC | 11:46 | |
| *** otter768 has joined #openstack-infra | 11:46 | |
| *** glauco has joined #openstack-infra | 11:47 | |
| *** thorst has joined #openstack-infra | 11:47 | |
| *** weshay has joined #openstack-infra | 11:48 | |
| *** fawadkhaliq has quit IRC | 11:48 | |
| *** yfried has quit IRC | 11:51 | |
| *** otter768 has quit IRC | 11:51 | |
| sdague | jhesketh: gotcha, too bad there isn't some monitoring on that :( we were just about to get to through the backlog | 11:51 |
| jhesketh | yeah :-( | 11:51 |
| *** coolsvap|away is now known as coolsvap | 11:51 | |
| jhesketh | sigh, apache/memcache is segfaulting.. this might put us back at the start of the queue | 11:53 |
| cdent | AJaeger: thanks for the reminder on the aodh functional job, I looked there before I saw the topic on irc channels | 11:53 |
| AJaeger | cdent: I was just afraid of a mass "recheck" from you ;) | 11:55 |
| cdent | nope, I got caught up quickly | 11:55 |
| AJaeger | (which might be completely ignored ;) | 11:55 |
| AJaeger | cdent: no worries | 11:55 |
| *** fawadkhaliq has joined #openstack-infra | 11:55 | |
| *** e0ne has quit IRC | 11:59 | |
| *** yamamoto has quit IRC | 12:02 | |
| *** pal has joined #openstack-infra | 12:03 | |
| openstackgerrit | Andrei V. Ostapenko proposed openstack-infra/project-config: Move from stackforge/magnetodb to openstack/magnetodb https://review.openstack.org/203524 | 12:04 |
| openstackgerrit | Andrei V. Ostapenko proposed openstack-infra/project-config: Changes jobs description for MagnetoDB https://review.openstack.org/204478 | 12:04 |
| *** pal has quit IRC | 12:07 | |
| *** glauco has quit IRC | 12:08 | |
| *** witek has joined #openstack-infra | 12:08 | |
| *** yfried has joined #openstack-infra | 12:09 | |
| *** tlian has joined #openstack-infra | 12:10 | |
| *** glauco_ has joined #openstack-infra | 12:10 | |
| *** _nadya_ has joined #openstack-infra | 12:12 | |
| *** woodster_ has joined #openstack-infra | 12:13 | |
| ttx | Sergey is in Mountain View those days, so we lost world coverage | 12:15 |
| *** erlon has joined #openstack-infra | 12:16 | |
| *** marcusvrn has joined #openstack-infra | 12:16 | |
| *** coolsvap is now known as coolsvap|away | 12:17 | |
| *** kiran-r has quit IRC | 12:18 | |
| *** fhubik is now known as fhubik_afk | 12:20 | |
| *** hashar has joined #openstack-infra | 12:21 | |
| jhesketh | ttx: I'm here, jsut debugging | 12:21 |
| *** gordc has joined #openstack-infra | 12:21 | |
| jhesketh | I can't figure out why after moving 15gb of content off the disk and syncing df still says the root vol is full | 12:21 |
| *** salv-orl_ has joined #openstack-infra | 12:22 | |
| jhesketh | I'm considering restarting zuul | 12:22 |
| *** glauco_ has quit IRC | 12:22 | |
| jhesketh | sdague: do you think a restart is reasonable? | 12:22 |
| AJaeger | jhesketh: open file handles on these files? Or moved data from the wrong partition? | 12:22 |
| sdague | jhesketh: what kind of files were moved? | 12:23 |
| jhesketh | AJaeger, sdague: I moved a bunch of logs that clarkb had copied for debugging onto another partition | 12:23 |
| sdague | open file handles is definitely the main culprit if df does not change | 12:23 |
| *** oomichi has joined #openstack-infra | 12:23 | |
| jhesketh | can't see any open file handles | 12:24 |
| *** salv-orlando has quit IRC | 12:25 | |
| *** e0ne has joined #openstack-infra | 12:25 | |
| sdague | so is fungi around? if so, I'd say wait. But he didn't seem to be about yesterday, in which case it's probably worth restarting and getting things rolling again | 12:25 |
| sdague | because there is a lot of backlog | 12:25 |
| *** salv-orlando has joined #openstack-infra | 12:26 | |
| *** salv-orl_ has quit IRC | 12:26 | |
| openstackgerrit | Emilien Macchi proposed openstack-infra/project-config: puppet: run copy_logs Rake task on beaker jobs https://review.openstack.org/206693 | 12:27 |
| jhesketh | ah, the partition I moved from was in a link so I didn't actually move across partitions | 12:27 |
| *** salv-orl_ has joined #openstack-infra | 12:27 | |
| jhesketh | there we go, freed up some space | 12:28 |
| *** vlaza has joined #openstack-infra | 12:29 | |
| *** salv-orlando has quit IRC | 12:30 | |
| AJaeger | will it recover itself? | 12:30 |
| *** hashar_ has joined #openstack-infra | 12:30 | |
| AJaeger | jhesketh: glad to hear! | 12:31 |
| *** dkranz has joined #openstack-infra | 12:31 | |
| jhesketh | hmm, not sure | 12:31 |
| jhesketh | it doesn't seem to be | 12:31 |
| jhesketh | but because the disk was full I don't have any logs | 12:31 |
| *** mjturek1 has quit IRC | 12:31 | |
| greghaynes | oh, at least its not another unknown fail :) | 12:31 |
| sdague | jhesketh: does sighup do a useful thing? | 12:31 |
| jhesketh | and it doesn't look like the zuul process has an open handle on the debug logs | 12:31 |
| jhesketh | sdague: just tried | 12:31 |
| jhesketh | hasn't seemed to | 12:31 |
| *** hashar has quit IRC | 12:33 | |
| sdague | yeh, restart I guess is good | 12:33 |
| *** deepakcs has quit IRC | 12:33 | |
| *** salv-orlando has joined #openstack-infra | 12:35 | |
| *** kgiusti has joined #openstack-infra | 12:37 | |
| *** aysyd has joined #openstack-infra | 12:38 | |
| *** bexelbie_ has joined #openstack-infra | 12:38 | |
| jhesketh | restart is in progress | 12:38 |
| *** bexelbie has quit IRC | 12:38 | |
| sdague | does nodepool need restarting as well? | 12:38 |
| *** salv-orl_ has quit IRC | 12:38 | |
| jhesketh | sdague: not nodepool, but the slaves | 12:38 |
| *** mpaolino has joined #openstack-infra | 12:38 | |
| jhesketh | so I'm going through those as well | 12:38 |
| sdague | ok | 12:38 |
| jhesketh | basically a slave needs to register with jenkins so that jenkins tells gearman it cna do the job | 12:38 |
| sdague | there is a ton of NOT_REGISTERED | 12:38 |
| sdague | ok | 12:38 |
| jhesketh | since gearman lost that state | 12:38 |
| jhesketh | so once new slaves are booted (which is happening now) | 12:39 |
| jhesketh | that'll fix it | 12:39 |
| *** mpavone has quit IRC | 12:39 | |
| *** vponomaryov has left #openstack-infra | 12:39 | |
| *** vponomaryov has joined #openstack-infra | 12:39 | |
| EmilienM | maybe we can send an IRC message to inform the community zuul was restarted | 12:41 |
| jhesketh | okay that's back | 12:42 |
| jhesketh | re-queuing the jobs | 12:42 |
| jhesketh | EmilienM: yep, will do once it's shiny | 12:42 |
| EmilienM | cool | 12:42 |
| *** zeih has joined #openstack-infra | 12:43 | |
| *** salv-orl_ has joined #openstack-infra | 12:44 | |
| *** notnownikki has joined #openstack-infra | 12:45 | |
| *** mjturek1 has joined #openstack-infra | 12:46 | |
| openstackgerrit | Merged openstack-infra/project-config: Normalize projects.yaml https://review.openstack.org/206883 | 12:46 |
| *** salv-orlando has quit IRC | 12:47 | |
| *** mpavone has joined #openstack-infra | 12:47 | |
| jhesketh | #status ok zuul's disks were at capacity. Space has been freed up and jobs are being re-queued. | 12:47 |
| openstackstatus | jhesketh: sending ok | 12:47 |
| *** salv-orl_ has quit IRC | 12:47 | |
| openstackgerrit | David Shrewsbury proposed openstack-infra/shade: Remove unused server functions https://review.openstack.org/206730 | 12:48 |
| *** coolsvap|away is now known as coolsvap | 12:48 | |
| greghaynes | jhesketh++ | 12:48 |
| *** salv-orlando has joined #openstack-infra | 12:48 | |
| *** bswartz has joined #openstack-infra | 12:49 | |
| *** yamamoto has joined #openstack-infra | 12:49 | |
| sc68cal | disk space isn't infinite!?!?!? | 12:49 |
| *** ChanServ changes topic to "Discussion of OpenStack Developer and Community Infrastructure | docs http://docs.openstack.org/infra/ | bugs https://storyboard.openstack.org/ | source https://git.openstack.org/cgit/openstack-infra/ | channel logs http://eavesdrop.openstack.org/irclogs/%23openstack-infra/" | 12:49 | |
| -openstackstatus- NOTICE: zuul's disks were at capacity. Space has been freed up and jobs are being re-queued. | 12:49 | |
| *** hamzy has joined #openstack-infra | 12:50 | |
| *** kgiusti has joined #openstack-infra | 12:50 | |
| *** mjturek1 has left #openstack-infra | 12:50 | |
| jhesketh | sc68cal: yeah, I thought that was what cloud was for too | 12:51 |
| greghaynes | I think that feature is coming next cycle | 12:51 |
| openstackstatus | jhesketh: finished sending ok | 12:52 |
| *** baoli has joined #openstack-infra | 12:52 | |
| sc68cal | :) | 12:53 |
| *** rfolco has joined #openstack-infra | 12:54 | |
| *** pal has joined #openstack-infra | 12:54 | |
| *** kushal has quit IRC | 12:55 | |
| *** jaosorior has joined #openstack-infra | 12:56 | |
| *** dims has quit IRC | 12:57 | |
| *** salv-orl_ has joined #openstack-infra | 12:57 | |
| *** annegentle has joined #openstack-infra | 12:57 | |
| *** dims has joined #openstack-infra | 12:57 | |
| *** mriedem has joined #openstack-infra | 12:57 | |
| openstackgerrit | David Shrewsbury proposed openstack-infra/shade: Remove get_openstack_vars() method https://review.openstack.org/207027 | 12:59 |
| *** jamespage_ has joined #openstack-infra | 12:59 | |
| *** pblaho has quit IRC | 13:00 | |
| *** gordc is now known as gordc_idle | 13:00 | |
| *** salv-orlando has quit IRC | 13:00 | |
| *** salv-or__ has joined #openstack-infra | 13:00 | |
| *** bknudson has quit IRC | 13:01 | |
| *** pblaho has joined #openstack-infra | 13:01 | |
| openstackgerrit | Anyi Li proposed openstack-infra/jenkins-job-builder: Add value description feature to extended choice parameter https://review.openstack.org/207028 | 13:01 |
| *** scheuran has quit IRC | 13:02 | |
| *** scheuran has joined #openstack-infra | 13:02 | |
| openstackgerrit | David Shrewsbury proposed openstack-infra/shade: Remove unused server functions https://review.openstack.org/206730 | 13:03 |
| openstackgerrit | David Shrewsbury proposed openstack-infra/shade: Remove get_openstack_vars() method https://review.openstack.org/207027 | 13:03 |
| EmilienM | jhesketh: what is taking so much place? | 13:04 |
| *** salv-orl_ has quit IRC | 13:04 | |
| *** jamespage__ has joined #openstack-infra | 13:04 | |
| *** jamespage_ has quit IRC | 13:05 | |
| *** dzamboni has joined #openstack-infra | 13:07 | |
| *** oomichi has quit IRC | 13:08 | |
| *** jsavak has joined #openstack-infra | 13:08 | |
| dzamboni | anteaya: ping | 13:08 |
| *** sigmavirus24_awa is now known as sigmavirus24 | 13:08 | |
| *** salv-or__ has quit IRC | 13:09 | |
| *** pblaho has quit IRC | 13:10 | |
| *** salv-orlando has joined #openstack-infra | 13:11 | |
| *** pblaho has joined #openstack-infra | 13:11 | |
| *** phschwartz has quit IRC | 13:11 | |
| *** k4n0 has quit IRC | 13:12 | |
| *** phschwartz has joined #openstack-infra | 13:12 | |
| *** pblaho has quit IRC | 13:12 | |
| *** glauco has joined #openstack-infra | 13:13 | |
| *** glauco has quit IRC | 13:14 | |
| *** coolsvap is now known as coolsvap|away | 13:14 | |
| *** salv-orl_ has joined #openstack-infra | 13:14 | |
| *** glauco has joined #openstack-infra | 13:14 | |
| *** erikwilson_ has joined #openstack-infra | 13:15 | |
| *** pblaho has joined #openstack-infra | 13:15 | |
| *** bknudson has joined #openstack-infra | 13:16 | |
| AJaeger | jhesketh: tahnks a lot! I'll send a green email now | 13:16 |
| *** pblaho has quit IRC | 13:17 | |
| *** salv-orlando has quit IRC | 13:17 | |
| *** alivigni has joined #openstack-infra | 13:17 | |
| jhesketh | EmilienM: possibly logs, but frankly the disk is just really small (40gb) | 13:17 |
| *** bexelbie_ is now known as bexelbie | 13:17 | |
| *** pblaho has joined #openstack-infra | 13:18 | |
| *** salv-orlando has joined #openstack-infra | 13:18 | |
| *** rfolco has quit IRC | 13:18 | |
| EmilienM | jhesketh: I though logs were store in remote storage, like elastic ? maybe could we also extend the ephemeral storage or create a persistent volume, attach it and mount the log dir on this new volume? | 13:19 |
| AJaeger | jhesketh: it looks like Zuul lost some events for changes that were queued during the stuck time. | 13:19 |
| *** pblaho has quit IRC | 13:19 | |
| jhesketh | EmilienM: I meant things like apache logs etc | 13:19 |
| AJaeger | Is there a way to requeue them - or should we ask for recheck/approval again? | 13:19 |
| *** pblaho has joined #openstack-infra | 13:19 | |
| jhesketh | AJaeger: ah, that's a good point, that would have happened | 13:19 |
| jhesketh | AJaeger: yeah, recheck will be necessary | 13:19 |
| AJaeger | jhesketh: please send a status notify that jobs since this morning (let's say 7:00 UTC) need to be requeued | 13:20 |
| *** amitgandhinz has joined #openstack-infra | 13:20 | |
| *** pblaho has quit IRC | 13:20 | |
| *** pblaho has joined #openstack-infra | 13:20 | |
| *** salv-orl_ has quit IRC | 13:21 | |
| jhesketh | AJaeger: is that when zuul stopped? | 13:21 |
| *** pblaho has quit IRC | 13:22 | |
| *** pblaho has joined #openstack-infra | 13:22 | |
| *** eharney has joined #openstack-infra | 13:23 | |
| *** jswarren has joined #openstack-infra | 13:23 | |
| AJaeger | jhesketh: I noticed it at 7:25 UTC | 13:23 |
| jhesketh | AJaeger: okay thanks | 13:24 |
| jhesketh | #status notice zuul jobs after about 07:00 UTC may need a 'recheck' to enter the queue. Look if your change is in http://status.openstack.org/zuul/ and recheck if not. | 13:25 |
| openstackstatus | jhesketh: sending notice | 13:25 |
| -openstackstatus- NOTICE: zuul jobs after about 07:00 UTC may need a 'recheck' to enter the queue. Look if your change is in http://status.openstack.org/zuul/ and recheck if not. | 13:26 | |
| *** akshai has joined #openstack-infra | 13:26 | |
| fungi | i'm around for a bit. anything i can help with? i just saw the update that things are flowing again? | 13:26 |
| jhesketh | fungi: things are good (as they can be) | 13:26 |
| jhesketh | some work for later on will be making sure the disk doesn't fill again | 13:26 |
| *** cdoron has joined #openstack-infra | 13:26 | |
| fungi | but yeah, this week is mostly me trying to move all my junk to the new house and babysitting contractors, so i've not been around much. sorry :/ | 13:27 |
| fungi | jhesketh: oh, we filled up a disk on zuul? or where? | 13:27 |
| jhesketh | fungi: zuul's root disk | 13:27 |
| openstackstatus | jhesketh: finished sending notice | 13:27 |
| fungi | debug logs getting much larger than before? | 13:28 |
| sdague | all the graphite metering seems bonkers | 13:28 |
| *** devkulkarni has joined #openstack-infra | 13:28 | |
| jhesketh | fungi: they are on a different partition | 13:29 |
| fungi | usually when graphite's gone toes-up, carbon-cache ran wild and got sniped by the kernel oom killer | 13:29 |
| *** BharatK has quit IRC | 13:29 | |
| fungi | jhesketh: interesting. so what was filling up zuul's root filesystem? | 13:29 |
| pabelanger | morning | 13:30 |
| *** yfried_ has joined #openstack-infra | 13:30 | |
| mriedem | lifeless: i'm confused about the caps on python-openstackclient in stable/kilo, g-r has it capped <1.1.0 https://github.com/openstack/requirements/blob/stable/kilo/global-requirements.txt#L135 but upper-constraints has it at ==1.5.0 https://github.com/openstack/requirements/blob/stable/kilo/upper-constraints.txt#L239 | 13:30 |
| mriedem | lifeless: looks like stable/kilo jobs only get osc 1.0.4 so upper-constraints appears to not work (which is probably good since it's invalid | 13:31 |
| *** rfolco has joined #openstack-infra | 13:31 | |
| *** zeih has quit IRC | 13:31 | |
| *** yfried has quit IRC | 13:31 | |
| jhesketh | fungi: not entirely sure... it's very small so it's likely been close for a while | 13:32 |
| *** jamespage__ has quit IRC | 13:32 | |
| jhesketh | apache logs maybe | 13:32 |
| fungi | jhesketh: interesting. i'll take a look at cacti when i switch computers in a bit | 13:32 |
| *** cdoron has quit IRC | 13:32 | |
| *** jamespage__ has joined #openstack-infra | 13:32 | |
| jhesketh | okay thanks | 13:32 |
| fungi | and yeah, if we've stepped up api calls to zuul (job status embedded in gerrit change screens turned on yet?) that could certainly cause a spike in apache logs | 13:33 |
| fungi | but even just an increase in people hitting status.o.o/zuul could do that | 13:33 |
| *** doug-fish has joined #openstack-infra | 13:33 | |
| sdague | fungi: job status in change screens is not on | 13:34 |
| openstackgerrit | Merged openstack-infra/project-config: Add jobs to support aodh functional testing https://review.openstack.org/205617 | 13:34 |
| fungi | okay, so yeah likely we've just been riding the line | 13:34 |
| sdague | however, pretty much everyone in openstack has the status page up this week, because the backlog has been so high, and they are trying to figure out where there stuff is | 13:34 |
| fungi | i suspect we need to work out a way to be able to tune log rotation through the apache puppet module | 13:35 |
| fungi | we've got similar situations with apache logs filling up root filesystems on our pypi mirrors and other places too | 13:35 |
| *** ayoung has joined #openstack-infra | 13:36 | |
| *** jamespage__ has quit IRC | 13:36 | |
| *** annegentle has quit IRC | 13:36 | |
| *** jamespage__ has joined #openstack-infra | 13:36 | |
| jhesketh | fungi: I'm not certain that's it. The apache logs are only 1.3gb | 13:37 |
| *** nelsnelson has joined #openstack-infra | 13:42 | |
| *** jamespage__ has quit IRC | 13:42 | |
| *** BharatK has joined #openstack-infra | 13:43 | |
| *** markvoelker has joined #openstack-infra | 13:44 | |
| *** pblaho has quit IRC | 13:45 | |
| *** pblaho has joined #openstack-infra | 13:45 | |
| openstackgerrit | Matt Riedemann proposed openstack-infra/elastic-recheck: Add query for trove stable/kilo job bug 1479358 https://review.openstack.org/207049 | 13:45 |
| openstack | bug 1479358 in Trove "gate-trove-functional-dsvm-mysql is broken on stable/kilo due to older openstackclient" [Undecided,Confirmed] https://launchpad.net/bugs/1479358 | 13:45 |
| openstackgerrit | li,chen proposed openstack-infra/project-config: A Devstack-plugin that installs a HDFS backend https://review.openstack.org/207050 | 13:45 |
| uvirtbot | Launchpad bug 1479358 in trove "gate-trove-functional-dsvm-mysql is broken on stable/kilo due to older openstackclient" [Undecided,Confirmed] https://launchpad.net/bugs/1479358 | 13:45 |
| uvirtbot | Launchpad bug 1479358 in trove "gate-trove-functional-dsvm-mysql is broken on stable/kilo due to older openstackclient" [Undecided,Confirmed] | 13:45 |
| *** pblaho has quit IRC | 13:47 | |
| *** pblaho has joined #openstack-infra | 13:47 | |
| *** otter768 has joined #openstack-infra | 13:47 | |
| *** jasondotstar has joined #openstack-infra | 13:48 | |
| *** teran has quit IRC | 13:49 | |
| *** pblaho has quit IRC | 13:49 | |
| *** pblaho has joined #openstack-infra | 13:49 | |
| *** glauco has quit IRC | 13:50 | |
| *** teran has joined #openstack-infra | 13:50 | |
| fungi | heh, uvirtbot came back again | 13:50 |
| *** glauco has joined #openstack-infra | 13:50 | |
| fungi | i thought we killed it? i'll add an akick entry | 13:50 |
| *** pblaho has quit IRC | 13:51 | |
| *** kushal has joined #openstack-infra | 13:52 | |
| *** otter768 has quit IRC | 13:52 | |
| *** salv-orlando has quit IRC | 13:52 | |
| *** ChanServ sets mode: +b *!~supybot@aluminium.linux2go.dk | 13:52 | |
| *** uvirtbot was kicked by ChanServ (Banned: no longer necessary) | 13:52 | |
| *** jasondot_ has quit IRC | 13:52 | |
| *** austin81 has joined #openstack-infra | 13:52 | |
| rcarrillocruz | SpamapS: heya | 13:53 |
| rcarrillocruz | you around yet? | 13:53 |
| *** fawadkhaliq has quit IRC | 13:53 | |
| *** katyafervent is now known as katyafervent_awa | 13:55 | |
| *** arxcruz has quit IRC | 13:55 | |
| *** glauco__ has joined #openstack-infra | 13:56 | |
| openstackgerrit | Matthew Treinish proposed openstack-infra/project-config: Add stable compat jobs to trove-integration repo https://review.openstack.org/207055 | 13:56 |
| *** fawadkhaliq has joined #openstack-infra | 13:56 | |
| mtreinish | SlickNik, mriedem: ^^^ | 13:56 |
| *** yamahata has joined #openstack-infra | 13:57 | |
| mtreinish | of course that assumes that the branch stuff was set up properly on that job definition | 13:57 |
| *** mwagner_lap has joined #openstack-infra | 13:58 | |
| mtreinish | I'll likely have to respin it | 13:58 |
| *** katyafervent_awa is now known as katyafervent | 13:58 | |
| *** devkulkarni has quit IRC | 13:59 | |
| *** glauco has quit IRC | 13:59 | |
| *** evrardjp has quit IRC | 13:59 | |
| *** fhubik_afk is now known as fhubik | 13:59 | |
| *** fawadkhaliq has quit IRC | 14:00 | |
| *** fawadkhaliq has joined #openstack-infra | 14:01 | |
| *** tonytan4ever has joined #openstack-infra | 14:01 | |
| *** bhunter71 has joined #openstack-infra | 14:02 | |
| yolanda | hi pabelanger, AJaeger, i'd like your thoughts here https://review.openstack.org/207058 | 14:03 |
| *** markvoelker has quit IRC | 14:04 | |
| *** anthonyper has quit IRC | 14:04 | |
| *** mhu has quit IRC | 14:04 | |
| openstackgerrit | Merged openstack-infra/elastic-recheck: Add query for trove stable/kilo job bug 1479358 https://review.openstack.org/207049 | 14:04 |
| openstack | bug 1479358 in Trove Integration "gate-trove-functional-dsvm-mysql is broken on stable/kilo due to older openstackclient" [Undecided,In progress] https://launchpad.net/bugs/1479358 - Assigned to Matt Riedemann (mriedem) | 14:04 |
| *** jd__ has quit IRC | 14:04 | |
| *** mescanef has quit IRC | 14:04 | |
| *** fbo has quit IRC | 14:04 | |
| *** GonZo2000 has quit IRC | 14:05 | |
| AJaeger | yolanda: LGTM, thanks | 14:05 |
| *** EmilienM has quit IRC | 14:05 | |
| *** evrardjp has joined #openstack-infra | 14:06 | |
| *** GonZo2000 has joined #openstack-infra | 14:06 | |
| *** GonZo2000 has quit IRC | 14:06 | |
| *** GonZo2000 has joined #openstack-infra | 14:06 | |
| spredzy | yolanda, AJaeger on this last review, what you did I thought about doing it here https://review.openstack.org/gitweb?p=openstack-infra/project-config.git;a=blob;f=jenkins/scripts/propose_msync_update.sh;h=d0ad61191129ecbba8270cbdd538f2ea375356a7;hb=1ae0d7d916113a0175ebe293c9c906552a90c492#l55 | 14:07 |
| spredzy | adding and if PROJECT == TheNameOfInfraProject ... | 14:07 |
| spredzy | With this review that means we do not simply share the hook, but also the requirements - they might differ | 14:08 |
| spredzy | We might not require the same gem, different version, have different Rake rules etc... | 14:09 |
| yolanda | yes, you are right with that | 14:09 |
| *** fbo has joined #openstack-infra | 14:09 | |
| *** EmilienM has joined #openstack-infra | 14:09 | |
| openstackgerrit | Matthew Treinish proposed openstack-infra/project-config: Add stable compat jobs to trove-integration repo https://review.openstack.org/207055 | 14:09 |
| *** anthonyper has joined #openstack-infra | 14:10 | |
| spredzy | So - what I had in mind - is that infra has it own modulesync project, Puppet Openstack has also its own they each have their own life but both project share the same post-merge hook | 14:10 |
| *** jd__ has joined #openstack-infra | 14:10 | |
| yolanda | we should share same way of syncing modules, but content of the repos may differ | 14:11 |
| AJaeger | jhesketh, fungi, looking at status.openstack.org/zuul the Job Stats graphs look strange, I miss them going up again. It looks like the system is idle. | 14:11 |
| *** alivigni has quit IRC | 14:12 | |
| *** ParsectiX has quit IRC | 14:12 | |
| spredzy | yolanda, http://paste.openstack.org/show/406239/ | 14:12 |
| jeblair | fungi, jhesketh: i may have contributed by uncompressing some logs to /tmp | 14:12 |
| *** yfried_ has quit IRC | 14:12 | |
| *** yfried_ has joined #openstack-infra | 14:13 | |
| yolanda | spredzy, idea looks good to me | 14:13 |
| yolanda | by this way we'll have more freedom on our config | 14:13 |
| *** alivigni has joined #openstack-infra | 14:13 | |
| spredzy | AJaeger, agree with that approach ? ^ | 14:13 |
| yolanda | pabelanger, when you come online, what do you think? ^ç | 14:14 |
| AJaeger | jeblair: I have a question for https://review.openstack.org/194595. This adds a new monasco repo. Should this go with other monasco repos to stackforge or move to openstack namespace? | 14:14 |
| *** annegentle has joined #openstack-infra | 14:14 | |
| AJaeger | spredzy: let me defer that to yolanda and pabelanger, they know the puppet modules and the syncing requirements far better than I do. | 14:15 |
| *** Sherif has joined #openstack-infra | 14:15 | |
| spredzy | yolanda, with that approach in mind would comment line 55 and 70 here https://review.openstack.org/#/c/189216/6/jenkins/scripts/propose_msync_update.sh,cm are still valid ? | 14:15 |
| spredzy | AJaeger, ack | 14:15 |
| yolanda | spredzy, for the projects check, maybe with some reorg | 14:16 |
| openstackgerrit | Timothy R. Chavez proposed openstack-infra/puppet-gerrit: Add support for 'projects' keyword https://review.openstack.org/206648 | 14:17 |
| yolanda | spredzy, instead of a conditional for each project, create some list with valid projects | 14:17 |
| openstackgerrit | Emilien Macchi proposed openstack-infra/project-config: puppet: run copy_logs Rake task on beaker jobs https://review.openstack.org/206693 | 14:18 |
| EmilienM | pabelanger: ^ can you look please ? | 14:18 |
| spredzy | yolanda, since I receive the name part and not the whole path (ie. puppet-modulesync-configs and not openstack/puppet-modulesync-configs), I still need to append the scrope in front of the name based on the project name (ie. openstack/ vs. openstack-infra/) | 14:19 |
| jhesketh | jeblair: ah okay | 14:19 |
| spredzy | So the I still need the if, no ? | 14:20 |
| SergeyLukjanov | sdague, sorry, I'm in CA now | 14:20 |
| openstackgerrit | Merged openstack/requirements: adding unicodecsv for writing and reading unicode csv. https://review.openstack.org/205613 | 14:20 |
| pabelanger | yolanda: is there a full patch to look at? | 14:20 |
| yolanda | pabelanger, we are talking about https://review.openstack.org/gitweb?p=openstack-infra/project-config.git;a=blob;f=jenkins/scripts/propose_msync_update.sh;h=d0ad61191129ecbba8270cbdd538f2ea375356a7;hb=1ae0d7d916113a0175ebe293c9c906552a90c492#l55 | 14:21 |
| *** markvoelker_ has joined #openstack-infra | 14:21 | |
| *** mhu has joined #openstack-infra | 14:22 | |
| yolanda | and https://review.openstack.org/#/c/207058/ | 14:22 |
| jeblair | i have restarted carbon-cache | 14:23 |
| *** vlaza has left #openstack-infra | 14:24 | |
| *** zz_dimtruck is now known as dimtruck | 14:24 | |
| pabelanger | yolanda: looking at puppet-msync, I cannot see a way to avoid it using git to pull down the repo: https://github.com/puppet-community/modulesync/blob/master/lib/modulesync.rb#L60 | 14:24 |
| pabelanger | so the clone happens twice | 14:24 |
| *** mtanino has joined #openstack-infra | 14:25 | |
| *** davidjc has joined #openstack-infra | 14:25 | |
| pabelanger | Ya, really not a fan of msync doing the cloning of projects | 14:26 |
| *** r-daneel has joined #openstack-infra | 14:26 | |
| *** mescanef has joined #openstack-infra | 14:26 | |
| spredzy | pabelanger, why is the clone happening twice ? | 14:27 |
| spredzy | infra clones puppet-modulesync-configs | 14:27 |
| spredzy | and then msync will clone the modules it manages | 14:27 |
| pabelanger | right | 14:27 |
| pabelanger | that is the issue I have | 14:27 |
| pabelanger | we loose all the functionality that zuul-clone provides | 14:27 |
| pabelanger | and duplicate functionality that propose_updates.sh has | 14:28 |
| spredzy | not familiar with zuul-clone, but which feature would you like to have here ? | 14:28 |
| pabelanger | give me a sec, going to get a patchset up explaining what I mean | 14:28 |
| spredzy | pabelanger, sure | 14:29 |
| fungi | zuul-cloner: http://docs.openstack.org/infra/zuul/cloner.html | 14:29 |
| *** glauco__ has quit IRC | 14:29 | |
| *** yamamoto has quit IRC | 14:29 | |
| *** yamamoto has joined #openstack-infra | 14:29 | |
| ttx | jeblair: following the discussion about lazy-vote changes at the TC yesterday, is there an easy way to set the topic on a review that you don't own ? | 14:30 |
| fungi | it provides us integration testing support across multiple repos (probably not relevant for a proposal job) and understanding how to make use of and update a git repo cache (definitely useful on single-use workers, not currently useful on our long-lived workers like proposal.slave.openstack.org though) | 14:30 |
| jeblair | pabelanger: i'm pretty sure we discussed and agreed that our approach would be: 1) manually sinc all of this stuff once 2) see if we need to make changes to those files often 3) if we do, then look into externalizing the content, like putting it in a gem | 14:31 |
| ttx | jeblair: I thought it was as easy as the commit message update from the UI but apparently not | 14:31 |
| jeblair | ttx: i did too | 14:31 |
| *** tvanderwiel has joined #openstack-infra | 14:31 | |
| *** baoli has quit IRC | 14:31 | |
| ttx | jeblair: if not I'll opt for a commit message update, that sounds less impactful than resubmitting | 14:31 |
| fungi | it's possible that's an acl? | 14:31 |
| fungi | disclaimer: i haven't looked | 14:32 |
| *** baoli has joined #openstack-infra | 14:32 | |
| ttx | You can certainly change yours if you own it | 14:32 |
| jeblair | ttx: i do it all the time, so the question is whether it's an admin leakage, and if so, can we control it with an acl | 14:32 |
| *** mpaolino has quit IRC | 14:32 | |
| * ttx looks | 14:32 | |
| ttx | edit topic name seems to be an ACL thing | 14:33 |
| jeblair | "Edit topic name" is a permission i can add | 14:33 |
| spredzy | fungi, ok read what it does but basically the feeling I have here is that we either use zuul-cloner + propose_update.sh to manage the sync of the files or either use modulesync has it mainly does the same thing. | 14:33 |
| ttx | maybe grant it to tc members (or just me) | 14:33 |
| fungi | jeblair: ttx: we should probably add that globally | 14:33 |
| jeblair | spredzy: see above. i am strongly opposed to developing any kind of "sync" infrastructure. | 14:33 |
| fungi | unless we want it to be limited to change owner and core reviewers per-project | 14:33 |
| AJaeger | it would help with marking projects as "project-rename" for the next bunch ;) | 14:34 |
| fungi | (like we do with wip and abandon) | 14:34 |
| ttx | fungi: I guess main drawback is that there is no audit trail | 14:34 |
| AJaeger | I agree with fungi. | 14:34 |
| jeblair | spredzy, pabelanger: the reason is because i do not think it is a good use of time for us to review individual tiny identical changes to many projects. | 14:34 |
| ttx | or is there ? | 14:34 |
| fungi | ttx: i believe it leaves a review comment | 14:35 |
| ttx | fungi: does that leave a message in the comments ? | 14:35 |
| ttx | fungi: only you can tell me | 14:35 |
| fungi | heh | 14:35 |
| jeblair | ttx: https://review.openstack.org/#/c/206795/ | 14:35 |
| jeblair | "Topic set to ..." | 14:35 |
| *** sabeen has joined #openstack-infra | 14:35 | |
| ttx | jeblair: ok, seems ok then | 14:35 |
| spredzy | jeblair, for the case of Puppet Openstack it will be around 14 projects (for now) with a topic name that would be quite easy to filter and review | 14:35 |
| spredzy | I mean tracking divergence across module is really time consuming | 14:36 |
| ttx | the only abuse i could think of would be to silently set offensive topics or play ping pong topics, but the audit trail solves that | 14:36 |
| spredzy | and then re-submitting a single review manually to all those projects is also time consuming | 14:36 |
| fungi | ttx: jeblair: yeah, basically anybody can already change the topic by uploading a new patchset, so blocking that in the webui is silly | 14:36 |
| ttx | jeblair: +1 to globally grant it | 14:36 |
| spredzy | when with that solution is reviewing a change and merge it if it is ok | 14:36 |
| jeblair | spredzy: yes; puppet-openstack and openstack-infra may have different goals here. i agree it is time consuming and dangerous. which is why we will see if these things really need to change first. | 14:37 |
| *** coolsvap|away is now known as coolsvap | 14:37 | |
| *** rossella_s has quit IRC | 14:37 | |
| jeblair | spredzy: but, in my opinion (which i feel strongly about for openstack-infra, but less strongly about for puppet-openstack) is that if many repos have identical files that change often, it's a design problem, and it needs to be redesigned so that information is not in that repo. a gem is ideal. | 14:37 |
| *** yamamoto has quit IRC | 14:38 | |
| *** rossella_s has joined #openstack-infra | 14:38 | |
| *** markvoelker_ has quit IRC | 14:38 | |
| yolanda | jeblair we talked a bit about that yesterday. There were some thoughts, about that relying in something that is not in-tree for testing that modules, was not a good idea | 14:38 |
| crinkle | a gem doesn't hold for things like beaker nodesets or .gitignore files | 14:38 |
| *** dkehn has quit IRC | 14:39 | |
| crinkle | but those are also less likely to change | 14:39 |
| *** markvoelker has joined #openstack-infra | 14:39 | |
| jeblair | spredzy: in python, every repo has a tox.ini. they are similar, but they are simple and don't change often (and also, don't actually need to be identical). it is used to bootstrap other things that need to be in common across all the repos | 14:39 |
| yolanda | i intially thought in pulling that stuff from project-config somehow,but there may be better ways | 14:39 |
| jeblair | crinkle: right -- we want to minimize it | 14:39 |
| *** koolhead17 has joined #openstack-infra | 14:40 | |
| jeblair | yolanda: yes, tests should still work outside our environment, so project-config (and even other git repos) aren't necesserily the best solution; something installable is better | 14:40 |
| *** sabeen3 has joined #openstack-infra | 14:41 | |
| *** yamahata has quit IRC | 14:41 | |
| *** soren has quit IRC | 14:41 | |
| *** sabeen has quit IRC | 14:42 | |
| jeblair | at any rate, let's please just see if this is a problem first. let's just merge those changes, do not merge any changes that just change these files on a single repo (it's easy to keep them from getting out of sync if you don't let them), and if we find that we have to change this more than once every 6 months or a year, we'll externalize it. | 14:42 |
| fungi | i agree, it seems like a premature optimization | 14:43 |
| spredzy | jeblair, I have the feeling that your concerns are more from an OPS point of view, if things move it should be externalized so one can install this (package,gem,whatever) on every nodes it needs. From my perspective I mean more to have consistency across modules, not specifically related to installation but to any file that can be shared | 14:43 |
| crinkle | jeblair: ++ | 14:43 |
| jeblair | spredzy: nope, it's from the point of view that it is a waste of time for reviewers to review identical changes to multiple repos | 14:43 |
| jeblair | we like to avoid reviewing trivial changes | 14:44 |
| *** mpaolino has joined #openstack-infra | 14:44 | |
| jeblair | and from a compsci/engineering point of view, it's basically the DRY principle | 14:44 |
| *** glauco_ has joined #openstack-infra | 14:45 | |
| *** pal has quit IRC | 14:45 | |
| *** pal has joined #openstack-infra | 14:46 | |
| AJaeger | jeblair, fungi: I have a question for https://review.openstack.org/194595. This adds a new monasco repo. Should this go with other monasco repos to stackforge or move to openstack namespace? - sorry for pinging again, I'll sign of soonish... | 14:46 |
| *** markvoelker has quit IRC | 14:47 | |
| *** markvoelker has joined #openstack-infra | 14:47 | |
| *** zeih has joined #openstack-infra | 14:47 | |
| jeblair | AJaeger: i would put it with the rest of them to make it easy on developers and users | 14:48 |
| jeblair | (in stackforge) | 14:48 |
| *** ZZelle has quit IRC | 14:48 | |
| AJaeger | jeblair: ok, then I +2 now. | 14:48 |
| jeblair | and we'll move them all together | 14:48 |
| fungi | jeblair: jhesketh: so... looking at cacti, i agree this was probably manually inflicted (by decompressing some logfiles or whatever). seems we jumped from ~50% to almost full in a matter of a day | 14:48 |
| AJaeger | jeblair: I wasn't sure whether we have a complete freeze on the namespace or not. | 14:48 |
| jeblair | fungi: yeah, sorry moving them to /opt now | 14:48 |
| AJaeger | jeblair: So, adding "independent" repos to openstack, but ok to add related to stackforge? | 14:49 |
| jeblair | AJaeger: yeah, for the moment, let's say "put new repos in openstack unless they are part of a project already existing in stackforge" | 14:49 |
| *** bexelbie has quit IRC | 14:49 | |
| fungi | AJaeger: yep, when we do a mass move, a few additional repos won't really make a difference to the workload. its going to have to be entirely scripted regardless | 14:49 |
| *** FallenPegasus has joined #openstack-infra | 14:50 | |
| AJaeger | jeblair, fungi: Works for me, thanks for the guidance! | 14:50 |
| *** pal has quit IRC | 14:50 | |
| *** zeih has quit IRC | 14:51 | |
| *** yamamoto has joined #openstack-infra | 14:54 | |
| *** esker has joined #openstack-infra | 14:54 | |
| spredzy | jeblair, I understand your point, but I might be missing something but the situation is the same today. What I mean is that if in puppet-openstack we need to update the Rakefile in all our modules, we will still have to submit N reviews, and our reviwers will need to review them. So I just don't understand why not having the hook, will not make reviewers review a change that need to applies globally to all our modules. | 14:54 |
| jeblair | spredzy: yes, that will still need to happen for that, and as crinkle mentioned, a few other files. but actual ruby code (like the spec helper) can be externalized if it changes often enough. | 14:56 |
| crinkle | (most of the Rakefile can be externalized as well if we need to) | 14:58 |
| *** jasondotstar has quit IRC | 14:59 | |
| *** tsekiyama has joined #openstack-infra | 14:59 | |
| spredzy | crinkle, just so I understand how would we do it ? Create a gem (with spec_accepance, Rakefile, etc..) that would be part of all puppet-openstack module's Gemfile ? | 14:59 |
| spredzy | So actually only the Gemfile remains the possible moving part not being synchronized? | 15:00 |
| *** jsavak has quit IRC | 15:00 | |
| *** jistr has quit IRC | 15:00 | |
| crinkle | spredzy: we'd still need a 'require' in the rakefile and spec helpers, so the files would still be there, but there would be a lot less code in them | 15:01 |
| *** jsavak has joined #openstack-infra | 15:01 | |
| *** k4n0 has joined #openstack-infra | 15:01 | |
| yolanda | crinkle, i was talking about creating some cookiecutter with all the repeated stuff, at least to create new puppet modules | 15:01 |
| *** e0ne has quit IRC | 15:02 | |
| jeblair | cookiecutter is a good idea | 15:02 |
| *** mgkwill has quit IRC | 15:02 | |
| *** edwarnicke has quit IRC | 15:02 | |
| *** sweston has quit IRC | 15:02 | |
| *** electrical has quit IRC | 15:02 | |
| *** dougwig has quit IRC | 15:02 | |
| *** mestery has joined #openstack-infra | 15:02 | |
| crinkle | yolanda: we use cookiecutter but it doesn't help for keeping things in sync after its created initially | 15:02 |
| crinkle | afaik | 15:02 |
| spredzy | yolanda, https://github.com/openstack/puppet-openstack-cookiecutter | 15:02 |
| spredzy | crinkle, nop it doesn't. It just generate common dynamic file. It is a one time job | 15:02 |
| *** jistr has joined #openstack-infra | 15:02 | |
| spredzy | to boostrap the modules | 15:02 |
| yolanda | crinkle, i know, and i had that fear as well | 15:02 |
| jeblair | crinkle: right, but is still a good way to define what a new project should look like | 15:02 |
| crinkle | yes | 15:03 |
| jeblair | (and is really easy to set up, so premature optimization arguments don't necessarily apply) | 15:03 |
| *** e0ne has joined #openstack-infra | 15:04 | |
| *** jsavak has quit IRC | 15:05 | |
| spredzy | crinkle, so taking the output of a noop run of msync puting all that in a gem and requiring this gem in our modules seems like a reasonable way to go ? | 15:06 |
| *** rlandy has joined #openstack-infra | 15:06 | |
| mriedem | is there any way we can bump https://review.openstack.org/#/c/206676/ in the zuul queue? | 15:06 |
| mriedem | blocking grenade jobs on master for the base integrated projects | 15:06 |
| spredzy | So when a change happens, a review needs to be made on all modules to specify a newer version | 15:06 |
| fungi | mriedem: is it likely to pass its jobs this time? | 15:07 |
| pabelanger | yolanda: spredzy: Here is the first part of the patch: http://pastebin.com/YPhciAUb | 15:07 |
| pabelanger | just tested it locally | 15:07 |
| mriedem | fungi: it's passing some jobs already, i think the things it failed on last night were some timeouts | 15:07 |
| mriedem | which were global | 15:07 |
| *** glauco_ has quit IRC | 15:07 | |
| pabelanger | spredzy: yolanda super ugly parsing yaml in bash, but you get the idea | 15:08 |
| pabelanger | the tools/sync-projects.sh would then only invoke msync to actually copy files into project, using templates if needed | 15:08 |
| yolanda | crinkle, you say you use cookiecutter, is that publicly on openstack-infra? | 15:08 |
| crinkle | spredzy: we have to do a little more refactoring than that but basically yes | 15:09 |
| pabelanger | have something for that locally, but I patched msync | 15:09 |
| crinkle | yolanda: yes we have a repo, spredzy linked it above | 15:09 |
| spredzy | yolanda, https://github.com/openstack/puppet-openstack-cookiecu | 15:09 |
| fungi | mriedem: done | 15:09 |
| spredzy | crinkle, hmm ok well I'll add it to next week agenda so we can discuss more then | 15:09 |
| *** nfedotov has quit IRC | 15:10 | |
| mriedem | fungi: thanks, didn't want to lose a day because of htis | 15:10 |
| mriedem | and people are already rechecking | 15:10 |
| spredzy | pabelanger, /me checking | 15:10 |
| pabelanger | spredzy: yolanda http://pastebin.com/3pzqG65K | 15:10 |
| pabelanger | fixes copy error | 15:10 |
| yolanda | crinkle, i miss specific infra bits there? | 15:10 |
| jeblair | pabelanger: i'd encourage you to read the backlog here to see why i am not in favor of that approach | 15:10 |
| *** mpaolino has quit IRC | 15:11 | |
| crinkle | yolanda: i'm not sure i understand your question, but spredzy set it up so he would be the one to ask | 15:12 |
| spredzy | yolanda, this is a template for puppet-openstack modules | 15:12 |
| yolanda | yes, i'd love to have same for openstack-infra bits | 15:12 |
| jeblair | yes, a separate cookiecutter repo would be needed for infra | 15:13 |
| yolanda | ok, we could create a project and work on it | 15:13 |
| Swanson | Stable/kilo CI broken? Never had every test fail before. And yesterday it wasn't even providing logs and today the logs look like a setup error. pbr.version.SemanticVersion seems a problem. Along with some failed to resolve errors. | 15:15 |
| AJaeger | Swanson: there's an email thread on openstack-dev with details | 15:16 |
| *** davideagnello has joined #openstack-infra | 15:16 | |
| *** gordc_idle has quit IRC | 15:16 | |
| *** flepied has quit IRC | 15:17 | |
| *** flepied has joined #openstack-infra | 15:17 | |
| Swanson | AJaeger: Just found it. What I get for breaking the mailing list out into cinder and "everything else". Thanks. I'll wait for the all clear before hitting recheck. | 15:20 |
| *** davideagnello has quit IRC | 15:20 | |
| sdague | so, any chance I can get another infra core to look at d-g changes for multinode grenade - https://review.openstack.org/#/c/199091/. It's a 3 patch series, tries to be easy to review, and would unstick some things that both the neutron and nova teams want to do on rolling upgrade testing. | 15:21 |
| openstackgerrit | Kyle Mestery proposed openstack-infra/project-config: dragonflow: Enable python3 jobs https://review.openstack.org/207093 | 15:22 |
| clarkb | and the diff looks bigger than it actually is (some code reorg to be reconsumable) | 15:22 |
| *** tsekiyama has quit IRC | 15:23 | |
| openstackgerrit | Evgeny Antyshev proposed openstack-infra/zuul: Dependencies from undefined projects https://review.openstack.org/207094 | 15:23 |
| *** hashar_ is now known as hashar | 15:23 | |
| greghaynes | mordred: some other folks are running in to https://review.openstack.org/#/c/204522/ | 15:24 |
| greghaynes | review plz :) | 15:24 |
| openstackgerrit | greghaynes proposed openstack-infra/glean: Dont fail if openssh is not installed https://review.openstack.org/204522 | 15:24 |
| *** tsekiyama has joined #openstack-infra | 15:25 | |
| openstackgerrit | Sean Dague proposed openstack-infra/nodepool: nodepool devstack plugin https://review.openstack.org/207095 | 15:26 |
| mordred | greghaynes: done | 15:26 |
| mordred | clarkb: could you +A the two outstanding glean patches real quick? | 15:26 |
| pabelanger | jeblair: sorry, missed backlog | 15:26 |
| *** jsavak has joined #openstack-infra | 15:27 | |
| *** k4n0 has quit IRC | 15:27 | |
| openstackgerrit | Merged openstack-infra/glean: Overwrite .ssh/authorized_keys rather than append https://review.openstack.org/179485 | 15:27 |
| rcarrillocruz | mordred: yep, just hit this today | 15:27 |
| *** AJaeger has quit IRC | 15:28 | |
| sdague | clarkb: as promised - https://review.openstack.org/#/c/207095/1 it's even partially tested, though I got to the point of not quite understanding nodepool config enough to figure out why image spawn didn't seem to work right | 15:28 |
| *** sputnik13 has joined #openstack-infra | 15:28 | |
| *** soren has joined #openstack-infra | 15:28 | |
| clarkb | mordred: I just +2'd the one greghaynes linked. I also commented on lolwhynossh but meh | 15:28 |
| rcarrillocruz | also, getting issues when defining two nics on network_info.json, poking at it now | 15:28 |
| sdague | but, it should be in a state that it's easy for someone to take it the last mile | 15:28 |
| greghaynes | clarkb: we should ssh too, but glean should also not fail | 15:28 |
| clarkb | sdague: thank you | 15:28 |
| clarkb | mordred: your +2 was reset so I think you can reapply and approve | 15:29 |
| mordred | clarkb: ossum | 15:29 |
| *** esmute_ has joined #openstack-infra | 15:30 | |
| greghaynes | yea, I was amazed at how badly I typed that commit msg and had to fix it | 15:30 |
| rcarrillocruz | mordred: btw, given that on ubuntu-minimal glean fails due to missing openssh-server, how you folks install openssh in the image? you use an element from tripleo-elements repo or some other | 15:30 |
| rcarrillocruz | ? | 15:30 |
| *** flepied has quit IRC | 15:31 | |
| clarkb | rcarrillocruz: iirc our puppet installs ssh | 15:31 |
| clarkb | rcarrillocruz: we have an entir essh module | 15:31 |
| mordred | rcarrillocruz: ^^ what clarkb said | 15:31 |
| mordred | we have additional elements in the project-config repo | 15:31 |
| mordred | that do things like setup puppet and whatnot | 15:31 |
| rcarrillocruz | ah | 15:31 |
| rcarrillocruz | so you puppet, kick off puppet from it | 15:31 |
| rcarrillocruz | interesting | 15:31 |
| rcarrillocruz | let me look | 15:31 |
| *** baoli has quit IRC | 15:31 | |
| *** banix has joined #openstack-infra | 15:31 | |
| mordred | yeah -for the nodepool instances, we actually run the puppet as part of dib | 15:32 |
| *** Sherif has quit IRC | 15:32 | |
| *** flepied has joined #openstack-infra | 15:33 | |
| openstackgerrit | Kyle Mestery proposed openstack-infra/project-config: dragonflow: Enable python3 jobs https://review.openstack.org/207093 | 15:33 |
| openstackgerrit | Merged openstack-infra/glean: Dont fail if openssh is not installed https://review.openstack.org/204522 | 15:34 |
| *** btully has quit IRC | 15:34 | |
| *** btully has joined #openstack-infra | 15:34 | |
| *** alexpilotti has quit IRC | 15:34 | |
| *** flepied has quit IRC | 15:38 | |
| *** gordc has joined #openstack-infra | 15:39 | |
| *** baoli has joined #openstack-infra | 15:39 | |
| *** esker has quit IRC | 15:39 | |
| clarkb | sdague: ya first glance at this look sgood, I will have to spin it up and figure out the image builds | 15:39 |
| openstackgerrit | Merged openstack-infra/shade: Fix two typos and one readablity on shade documentation https://review.openstack.org/206871 | 15:42 |
| *** dannywilson has joined #openstack-infra | 15:42 | |
| mordred | clarkb: can you look at https://review.openstack.org/#/c/202250/ ? I'd like to cut a new glean release, but think we should go ahead and merge that in first | 15:42 |
| *** fhubik is now known as fhubik_afk | 15:42 | |
| openstackgerrit | Merged openstack-infra/subunit2sql: Adds method to get metadata from a run https://review.openstack.org/206192 | 15:43 |
| *** harlowja_at_home has joined #openstack-infra | 15:44 | |
| *** fhubik_afk is now known as fhubik | 15:45 | |
| *** devvesa has joined #openstack-infra | 15:45 | |
| sdague | clarkb: there were times when it looked like it was starting a compute, and it was running, but it didn't seem to see it | 15:46 |
| sdague | my base testing env isn't neutron, so that might have been a thing | 15:46 |
| *** Sukhdev_ has joined #openstack-infra | 15:47 | |
| clarkb | mordred: I like that the debian case is so much simpler >_> | 15:47 |
| *** otter768 has joined #openstack-infra | 15:48 | |
| mordred | clarkb: :) | 15:48 |
| *** spredzy is now known as spredzy|afk | 15:49 | |
| *** esker has joined #openstack-infra | 15:50 | |
| openstackgerrit | yolanda.robla proposed openstack-infra/project-config: Add openstack-infra/puppet-cookiecutter project https://review.openstack.org/207109 | 15:50 |
| *** esker has quit IRC | 15:50 | |
| *** ifarkas has quit IRC | 15:50 | |
| mriedem | would be nice to get another review on this https://review.openstack.org/#/c/183124/ - it continues to be an issue in the gate | 15:52 |
| *** vinay_yadhav has quit IRC | 15:52 | |
| *** otter768 has quit IRC | 15:52 | |
| *** patrickeast has joined #openstack-infra | 15:53 | |
| *** esker has joined #openstack-infra | 15:54 | |
| *** esker has quit IRC | 15:54 | |
| *** fawadkhaliq has quit IRC | 15:54 | |
| *** shashank_hegde has joined #openstack-infra | 15:56 | |
| *** doude has quit IRC | 15:59 | |
| *** jsavak has quit IRC | 15:59 | |
| erlon | . | 16:00 |
| *** jlanoux has quit IRC | 16:01 | |
| *** baoli has quit IRC | 16:01 | |
| openstackgerrit | Merged openstack-infra/glean: Support explicitly configured DHCP interfaces https://review.openstack.org/202250 | 16:01 |
| *** esker has joined #openstack-infra | 16:01 | |
| *** jsavak has joined #openstack-infra | 16:02 | |
| *** baoli_ has joined #openstack-infra | 16:03 | |
| clarkb | mriedem: looking now (I still want us to stop pruning but we can deal with that later) | 16:03 |
| *** soren has quit IRC | 16:04 | |
| openstackgerrit | Emilien Macchi proposed openstack-infra/project-config: puppet: run copy_logs Rake task on beaker jobs https://review.openstack.org/206693 | 16:04 |
| *** dtantsur is now known as dtantsur|afk | 16:04 | |
| clarkb | mriedem: we could even do `retry_cmd` and make it less git specific. Ok I am done making suggestions that don't help solve the immediate problem | 16:05 |
| *** flepied has joined #openstack-infra | 16:06 | |
| *** jistr has quit IRC | 16:08 | |
| mriedem | clarkb: thanks | 16:08 |
| mriedem | clarkb: we'll do retry_cmd when we create oslo.devstack and just move this common stuff there :) | 16:08 |
| *** sdake has joined #openstack-infra | 16:08 | |
| mriedem | oslo.bash? | 16:09 |
| dims | haha | 16:09 |
| *** ihrachyshka has quit IRC | 16:09 | |
| jeblair | :) [actually most of this will be replaced by zuul-cloner once someone gets around to that] | 16:10 |
| *** electrical has joined #openstack-infra | 16:10 | |
| *** jaypipes has joined #openstack-infra | 16:12 | |
| *** coolsvap is now known as coolsvap|away | 16:12 | |
| *** puranamr has joined #openstack-infra | 16:13 | |
| *** ctracey has quit IRC | 16:13 | |
| *** briancurtin has quit IRC | 16:13 | |
| *** zhiyan has quit IRC | 16:13 | |
| *** ameade has quit IRC | 16:13 | |
| *** xyang has quit IRC | 16:13 | |
| *** nzeer has quit IRC | 16:13 | |
| *** simonmcc has quit IRC | 16:13 | |
| *** thingee has quit IRC | 16:13 | |
| *** jraim has quit IRC | 16:13 | |
| *** nimrodsun has quit IRC | 16:13 | |
| *** devvesa has quit IRC | 16:13 | |
| lifeless | mriedem: the constraints files are not relevant to kilo or juno | 16:13 |
| lifeless | mriedem: it being added at all was an oversight | 16:13 |
| mriedem | lifeless: yeah i see it was just part of that giant backport | 16:13 |
| mriedem | lifeless: want to remove it? | 16:14 |
| lifeless | I think we sould yeah | 16:14 |
| *** glauco has joined #openstack-infra | 16:14 | |
| lifeless | unless/until someone signs up to backport the *use of constraints* | 16:14 |
| *** _nadya_ has quit IRC | 16:14 | |
| lifeless | which is itself quite a project | 16:14 |
| *** FallenPegasus has quit IRC | 16:15 | |
| *** _nadya_ has joined #openstack-infra | 16:15 | |
| *** kun_huang has left #openstack-infra | 16:15 | |
| *** harlowja_at_home has quit IRC | 16:15 | |
| clarkb | +1 | 16:16 |
| *** edwarnicke has joined #openstack-infra | 16:16 | |
| fungi | agreed, we should remove the constraints file from stable branches of the reqs repo, at least until such time as someone works on backporting constraints support in devstack et al | 16:17 |
| *** unicell has quit IRC | 16:17 | |
| *** xyang1 has joined #openstack-infra | 16:18 | |
| fungi | and the constraints proposal job is at least already set up to work for stable/liberty once that exists, proven by its earlier proposals to stable/kilo | 16:18 |
| *** david-lyle has quit IRC | 16:18 | |
| *** ashleighfarnham has joined #openstack-infra | 16:19 | |
| lifeless | fungi: so devstack isn't the hard bit: its updating the pbr constraints and releasing *everything from that release again* | 16:19 |
| *** _nadya_ has quit IRC | 16:19 | |
| fungi | speaking of which, now would also be a good time for someone to approve https://review.openstack.org/205292 | 16:19 |
| *** rlandy has quit IRC | 16:19 | |
| lifeless | fungi: then making the global requirments py2+py3 ready, and releasing everything *again* | 16:20 |
| fungi | lifeless: ahh, yep, agreed | 16:20 |
| SpamapS | rcarrillocruz: I"m here now | 16:20 |
| lifeless | (which is more-or-less the process needed) | 16:20 |
| *** rlandy has joined #openstack-infra | 16:20 | |
| *** fhubik is now known as fhubik_afk | 16:22 | |
| *** nzeer has joined #openstack-infra | 16:23 | |
| *** ivar-lazzaro has joined #openstack-infra | 16:23 | |
| jeblair | clarkb, fungi, jhesketh: aha! i have greatly narrowed down the zuul problem! the reconfiguration _failed_ but since it is invoked in a signal handler, the exception was not logged. | 16:23 |
| *** esmute_ has quit IRC | 16:24 | |
| lifeless | jeblair: how did you determine that, if it wasn't logged? | 16:24 |
| fungi | timing? | 16:24 |
| jeblair | lifeless: absence of log messages :) | 16:24 |
| lifeless | jeblair: oh, so you expected log A, B, C from reconfig, but observed A and B but not C ? | 16:24 |
| lifeless | jeblair: or something like that? | 16:24 |
| openstackgerrit | Clint 'SpamapS' Byrum proposed openstack-infra/shade: Document create_object https://review.openstack.org/206872 | 16:24 |
| jeblair | lifeless: yes -- i saw that it was re-enqueing the check queues, and then realized i did _not_ see it re-enqueing the gate queues... and then realized the "reconfiguration complete" message was missing to boot. | 16:25 |
| *** yfried_ has quit IRC | 16:25 | |
| lifeless | jeblair: nice sleuthing | 16:25 |
| *** abregman has quit IRC | 16:25 | |
| jeblair | thanks | 16:25 |
| lifeless | jeblair: btw is that a literal signal handler doing a chunk of work, or something scheduled *by* a signal handler ? | 16:26 |
| jeblair | so next step is to log those exceptions... i'll stare at it a bit more and see if i can't predict a more proximal cause... | 16:26 |
| *** jraim has joined #openstack-infra | 16:26 | |
| *** afazekas has quit IRC | 16:27 | |
| jeblair | lifeless: it schedules another thread to do the work, but then waits | 16:28 |
| *** dougwig has joined #openstack-infra | 16:29 | |
| lifeless | jeblair: ok, so its probably not an issue here, but that makes me worry a little about double-entry into signal processing | 16:29 |
| *** markus_z has quit IRC | 16:29 | |
| lifeless | by not an issue here, I mean its probably not the bug you're investigating | 16:30 |
| jeblair | lifeless: i think it disables/enables that signal | 16:30 |
| lifeless | zuul right? | 16:30 |
| jeblair | lifeless: yep | 16:30 |
| lifeless | I'll have a quick look | 16:30 |
| eantyshev | jeblair: Hi! there is my change https://review.openstack.org/#/c/148926 | 16:30 |
| jeblair | lifeless: zuul/cmd/server.py and it references methods in zuul/scheduler.py | 16:30 |
| jeblair | eantyshev: i thought this sounded familiar :) | 16:31 |
| eantyshev | jeblair: it's a fix for something like you described here:) | 16:31 |
| *** koolhead17 has quit IRC | 16:32 | |
| jeblair | eantyshev: yep, i think that's exactly it | 16:32 |
| lifeless | jeblair: that looks fine indeed, cool. | 16:32 |
| *** ihrachyshka has joined #openstack-infra | 16:32 | |
| jeblair | eantyshev: i was just going to look for that change because i thought i remembered reading it, thank you very much :) | 16:33 |
| lifeless | jeblair: though perhaps a different signal to SIGHUP would be good (aesthetics around hup actually killing things IFF stdin is still connected :) | 16:33 |
| *** sweston has joined #openstack-infra | 16:33 | |
| *** tonytan4ever has quit IRC | 16:34 | |
| jeblair | lifeless: yeah, though usr1 and usr2 are spoken for as well; though honestly, we have a gearman, we should just make these client commands. :) | 16:34 |
| openstackgerrit | Doug Hellmann proposed openstack/requirements: Clean up unit test https://review.openstack.org/205293 | 16:34 |
| openstackgerrit | Doug Hellmann proposed openstack/requirements: Require all requirements to be constrained or blacklisted https://review.openstack.org/204198 | 16:34 |
| openstackgerrit | Doug Hellmann proposed openstack/requirements: Move data validation out of the unit tests https://review.openstack.org/205290 | 16:34 |
| *** fhubik_afk is now known as fhubik | 16:34 | |
| jeblair | SUBMIT_JOB zuul:reconfigure | 16:34 |
| lifeless | jeblair: that would be nice quite independently of the signal | 16:35 |
| *** simonmcc has joined #openstack-infra | 16:35 | |
| lifeless | jeblair: having the signal as a backup might be useful (e.g. if its speaking to a dead gear or something) | 16:35 |
| *** shashank_hegde has quit IRC | 16:35 | |
| fungi | the old school bsd'er in me likes sighup for reconfigure, but i agree for a daemon with its own api interface that's less necessary | 16:35 |
| *** nimrodsun has joined #openstack-infra | 16:35 | |
| *** Sherif has joined #openstack-infra | 16:36 | |
| eantyshev | jeblair: I also found the other Zuul problem underneath my reconfiguration failures, which I strive to fix in https://review.openstack.org/#/c/207094 Please, take a look when you have time | 16:36 |
| *** sdake has quit IRC | 16:37 | |
| *** ctracey has joined #openstack-infra | 16:37 | |
| *** shashank_hegde has joined #openstack-infra | 16:38 | |
| lifeless | jeblair: we could probably use SIGRTMIN quite happily | 16:38 |
| *** coolsvap|away is now known as coolsvap | 16:39 | |
| lifeless | jeblair: though I haven't checked if Python exposes them today | 16:39 |
| lifeless | ('Unlike standard signals, real-time signals have no predefined | 16:39 |
| lifeless | meanings: the entire set of real-time signals can be used for | 16:39 |
| lifeless | application-defined purposes.') | 16:39 |
| jeblair | convenient :) | 16:39 |
| lifeless | there's uhm, 30 odd :) | 16:40 |
| lifeless | python exposes them | 16:40 |
| lifeless | ok, enough of this diversion, context switch time | 16:40 |
| *** david-lyle has joined #openstack-infra | 16:42 | |
| jeblair | eantyshev: that one is trickier; upstream, we want to remove changes for projects that don't exist | 16:42 |
| *** ameade has joined #openstack-infra | 16:43 | |
| jeblair | eantyshev: oh, but you still do remove the change... | 16:43 |
| jeblair | eantyshev: how does that fix your error then, i may not understand | 16:44 |
| *** sshnaidm has quit IRC | 16:44 | |
| eantyshev | jeblair: this fixes reconfiguration, which don't tolerate None as a project | 16:45 |
| *** sergmelikyan has joined #openstack-infra | 16:45 | |
| jeblair | eantyshev: do you have a traceback from that? because i think it should work | 16:45 |
| *** alexpilotti has joined #openstack-infra | 16:45 | |
| jeblair | eantyshev: oh i see | 16:45 |
| jeblair | eantyshev: it's "item.change.project.name" that was causing an attributeerror, right? | 16:45 |
| *** esmute_ has joined #openstack-infra | 16:46 | |
| *** zhiyan has joined #openstack-infra | 16:46 | |
| jeblair | (but that's still not fixed in the patch) | 16:46 |
| eantyshev | jeblair: perhaps I should make another patch with unittest exposing the problem | 16:47 |
| *** baoli_ has quit IRC | 16:48 | |
| jeblair | eantyshev: that would be great, thanks | 16:48 |
| jeblair | eantyshev: i'll leave some comments on that patch while i'm thinking about it | 16:48 |
| *** baoli_ has joined #openstack-infra | 16:49 | |
| *** nikeshm__ has joined #openstack-infra | 16:49 | |
| *** sergmelikyan has quit IRC | 16:49 | |
| nikeshm__ | hi | 16:50 |
| *** unicell has joined #openstack-infra | 16:50 | |
| *** Somay has joined #openstack-infra | 16:51 | |
| dzamboni | krtaylor: ping | 16:52 |
| docaedo | Hi, for the app-catalog project we have a horizon plugin and want to get a new repository for the horizon bits. I see a patch for lbaas dashboard that looks like what I would need to do (https://review.openstack.org/#/c/202281) but I'm guessing there's more to it? | 16:52 |
| *** AzherKhan has joined #openstack-infra | 16:52 | |
| jeblair | docaedo: http://docs.openstack.org/infra/manual/creators.html | 16:52 |
| *** laughterwym has quit IRC | 16:53 | |
| jeblair | docaedo: yeah, that's probably a good template patch, but there's the docs that explain it all | 16:53 |
| *** sdake has joined #openstack-infra | 16:53 | |
| *** tlian has quit IRC | 16:53 | |
| *** BobBall is now known as BobBall_AWOL | 16:54 | |
| *** Ala has quit IRC | 16:54 | |
| docaedo | jeblair: thanks so much :) I swear I looked around for a doc that would tell me what to do first | 16:54 |
| openstackgerrit | Alan Pevec proposed openstack-infra/release-tools: Ironic participates in stable point releases starting with Kilo https://review.openstack.org/203774 | 16:55 |
| jeblair | i'm going to open a rax ticket with today's 48 stuck-in-delete servers | 16:56 |
| *** vinay_yadhav has joined #openstack-infra | 16:56 | |
| *** mgkwill has joined #openstack-infra | 16:56 | |
| jeblair | if anyone has any idea how to stop the cycle of me opening a support ticket every day to get servers deleted, i am all ears. | 16:57 |
| clarkb | jroll ^ | 16:57 |
| *** briancurtin has joined #openstack-infra | 16:57 | |
| *** skylerberg has joined #openstack-infra | 16:57 | |
| *** hashar has quit IRC | 16:57 | |
| clarkb | I wonder if john the tuba guy can help | 16:57 |
| clarkb | seems like a nova bug ya? | 16:58 |
| *** coolsvap is now known as coolsvap|away | 16:58 | |
| jeblair | johnthetubaguy: can help? | 16:58 |
| *** coolsvap|away is now known as coolsvap | 16:58 | |
| johnthetubaguy | hey, this is a cells v1 bug | 16:58 |
| johnthetubaguy | it totally slipped through testing | 16:58 |
| jroll | ninja'd. | 16:58 |
| johnthetubaguy | I think alaski we know how we are looking on that | 16:59 |
| johnthetubaguy | jroll: have you heard how thats going? | 16:59 |
| jeblair | johnthetubaguy: oh good. for what it's worth, my daily routine now involves opening up a ticket with rax to delete the approx 50-100 instances we have stuck in delete. i have most of that scripted now, which is better for me and less better for the fanatical support folks. :| | 16:59 |
| jroll | johnthetubaguy: alaski would know best, all I know is people are looking at it | 16:59 |
| *** tlian has joined #openstack-infra | 17:00 | |
| vipul | how does one get a project added to https://launchpad.net/openstack .. would be nice to be able to find bps from gerrit ;) | 17:00 |
| *** davideagnello has joined #openstack-infra | 17:00 | |
| clarkb | well wedont really have testing of cells... | 17:00 |
| johnthetubaguy | jeblair: yeah, its a pain, we just deployed out the latest code, and yeah, super nasty bug | 17:00 |
| johnthetubaguy | clarkb: we do now, but not this case it would see | 17:00 |
| clarkb | we exclude tge vast majoroty of tempest | 17:01 |
| clarkb | there is a great big regex of dont run these tests testr | 17:01 |
| alaski | clarkb: can you send me a couple of uuids and I can look to see what the actual issue is | 17:01 |
| johnthetubaguy | clarkb: there is too much in that list, but its mostly because those features don't work with cells v1 | 17:01 |
| openstackgerrit | Evgeny Antyshev proposed openstack-infra/zuul: Dependencies from undefined projects https://review.openstack.org/207094 | 17:01 |
| clarkb | alaski jeblair has them | 17:01 |
| *** pal has joined #openstack-infra | 17:02 | |
| jeblair | alaski: i just opened ticket 150729-ord-0001072 with a full list | 17:02 |
| alaski | jeblair: okay. I rarely touch the ticketing system but I'll see if I can dig that up | 17:02 |
| jeblair | alaski: oh i can send you a couple, 1 sec | 17:02 |
| *** e0ne has quit IRC | 17:03 | |
| *** thingee_ has joined #openstack-infra | 17:04 | |
| eantyshev | jeblair: I've uploaded another patchset for change 207094, unittest provided | 17:05 |
| *** tjones has joined #openstack-infra | 17:06 | |
| *** scheuran has quit IRC | 17:10 | |
| *** AzherKhan has quit IRC | 17:10 | |
| *** jschwarz has quit IRC | 17:11 | |
| *** mpavone has quit IRC | 17:11 | |
| *** jsavak has quit IRC | 17:11 | |
| *** Somay has quit IRC | 17:12 | |
| *** Volundr is now known as whoops | 17:12 | |
| *** jsavak has joined #openstack-infra | 17:12 | |
| pabelanger | Working on getting fedora22 dibs going for the gate, any core mind reviewing https://review.openstack.org/#/c/203729/ to bump ntp to fix an issue under fedora? | 17:13 |
| pabelanger | nibalizer: mind looking at https://review.openstack.org/#/c/202884/ patchset and +A? | 17:14 |
| *** salv-orlando has joined #openstack-infra | 17:14 | |
| *** alexpilotti has quit IRC | 17:14 | |
| *** FallenPegasus has joined #openstack-infra | 17:15 | |
| *** FallenPe_ has joined #openstack-infra | 17:16 | |
| *** _nadya_ has joined #openstack-infra | 17:16 | |
| nibalizer | sure | 17:18 |
| *** ajmiller has quit IRC | 17:18 | |
| *** ajmiller has joined #openstack-infra | 17:19 | |
| *** xyang has joined #openstack-infra | 17:19 | |
| *** FallenPegasus has quit IRC | 17:19 | |
| *** salv-orlando has quit IRC | 17:20 | |
| *** ybathia has joined #openstack-infra | 17:21 | |
| lifeless | vipul: you add them bottom up | 17:22 |
| lifeless | vipul: in your project details on lp | 17:22 |
| *** sobersabre has joined #openstack-infra | 17:22 | |
| vipul | lifeless: ah | 17:22 |
| *** salv-orlando has joined #openstack-infra | 17:22 | |
| vipul | thanks that makes more sense | 17:23 |
| *** patrickeast has quit IRC | 17:23 | |
| *** alexpilotti has joined #openstack-infra | 17:23 | |
| ybathia | horizon stable/juno run_tests.sh gave me oslo related unit test failure and I see horizon stable/juno requirements.txt and the global-requirements.txt in the requirements repo are different, submitted this path of fix the tests https://review.openstack.org/#/c/203783/ but jenkins is giving a strange error, does anyone know what am I missing here? | 17:25 |
| *** fhubik has quit IRC | 17:25 | |
| *** ihrachyshka has quit IRC | 17:25 | |
| clarkb | ybathia I think that job is in the process of being fixed on juno | 17:26 |
| *** achanda has joined #openstack-infra | 17:27 | |
| *** mjturek1 has joined #openstack-infra | 17:29 | |
| mordred | clarkb, greghaynes: just pushed 1.4.0 tag for glean | 17:30 |
| greghaynes | mordred: fun | 17:31 |
| *** tonytan4ever has joined #openstack-infra | 17:31 | |
| lifeless | mordred: mail you might want to reply to from me on -dev | 17:32 |
| mordred | lifeless: I think I'm fine with it if we actually get a good error message | 17:34 |
| *** esmute_ has quit IRC | 17:34 | |
| *** devkulkarni has joined #openstack-infra | 17:34 | |
| lifeless | mordred: nono, 'better' :) | 17:35 |
| mordred | lifeless: :) | 17:35 |
| lifeless | I'll do some tests and report back if thats the key thing | 17:35 |
| *** HeOS has quit IRC | 17:36 | |
| *** armax has joined #openstack-infra | 17:36 | |
| *** patrickeast has joined #openstack-infra | 17:38 | |
| *** FallenPe_ has quit IRC | 17:39 | |
| *** ihrachyshka has joined #openstack-infra | 17:39 | |
| openstackgerrit | David Shrewsbury proposed openstack-infra/shade: Remove meta.get_server_public_ip() function https://review.openstack.org/207139 | 17:40 |
| *** jaypipes has quit IRC | 17:40 | |
| *** pelix has quit IRC | 17:41 | |
| *** r-daneel has quit IRC | 17:44 | |
| *** esmute_ has joined #openstack-infra | 17:45 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/shade: Make client constructor calls consistent https://review.openstack.org/191765 | 17:46 |
| *** dannywilson has quit IRC | 17:46 | |
| *** Sukhdev_ has quit IRC | 17:47 | |
| jeblair | Exception: Unable to run ready script | 17:47 |
| jeblair | command: cd /opt/nodepool-scripts && ./configure_mirror.sh devstack-centos7-hpcloud-b5-4031600 | 17:47 |
| jeblair | sed: can't read /etc/pip.conf: No such file or directory | 17:47 |
| jeblair | that mean anything to anyone ^ | 17:47 |
| dhellmann | lifeless: guh, I messed up and amended the wrong commit in the chain for https://review.openstack.org/#/c/205290/ -- should I go clean that up, or can we treat the series as a series? :-/ | 17:47 |
| jeblair | clarkb, fungi, mordred, ianw: ^ ? | 17:47 |
| *** sergmelikyan has joined #openstack-infra | 17:48 | |
| mordred | jeblair: well ... we were sedding pip.conf to work around something a while back | 17:48 |
| mordred | I thought we didn't need to do that anymore | 17:48 |
| clarkb | jeblair we update pip.conf to use the cloud local mirror. I am guessing that image stopped wtiting the default pip.conf | 17:48 |
| clarkb | we do need to do it | 17:48 |
| mordred | oh. yah | 17:48 |
| *** notnownikki has quit IRC | 17:49 | |
| jeblair | it is affecting multiple image types | 17:49 |
| *** otter768 has joined #openstack-infra | 17:49 | |
| fungi | which ones? deb-based platforms too? | 17:49 |
| *** esmute_ has quit IRC | 17:49 | |
| *** notnownikki has joined #openstack-infra | 17:49 | |
| jeblair | possibly all of them | 17:49 |
| fungi | guessing we merged some puppet change in the past 24 hours which is appearing in today's images | 17:50 |
| jeblair | (bare|devstack)-(trusty|centos7|centos6|f21)-(rax|hpcloud) | 17:50 |
| jeblair | just skimming what's on my screen | 17:50 |
| fungi | looking to see what | 17:50 |
| jeblair | i will prepare to delete the images, but not execute | 17:51 |
| fungi | https://review.openstack.org/166252 | 17:52 |
| openstackgerrit | Matthew Treinish proposed openstack-infra/project-config: Add stable compat jobs to trove-integration repo https://review.openstack.org/207055 | 17:52 |
| *** puranamr has quit IRC | 17:52 | |
| fungi | that looks like the most likely offender from yesterday | 17:52 |
| *** Sherif has quit IRC | 17:52 | |
| fungi | yolanda: ^ insights? | 17:52 |
| *** pvaneck has joined #openstack-infra | 17:53 | |
| mtreinish | anteaya: ^^^ I think I'm still missing something in that patch | 17:53 |
| mtreinish | I'm not sure what though | 17:53 |
| mordred | fungi: I agree with you on that patch | 17:54 |
| *** otter768 has quit IRC | 17:54 | |
| jeblair | fungi: i'm going to start deleting images | 17:54 |
| jeblair | sound good? | 17:54 |
| lifeless | dhellmann: perhaps just squash them all together | 17:54 |
| fungi | jeblair: yes | 17:54 |
| lifeless | dhellmann: I don't think we gain a lot by having a chain for post-commit-fixups | 17:54 |
| *** puranamr has joined #openstack-infra | 17:54 | |
| dhellmann | lifeless: ok, cool, I'll do that | 17:54 |
| clarkb | jeblair yes | 17:54 |
| mordred | fungi: although it looks like it shudl install a file | 17:54 |
| *** jasondotstar has joined #openstack-infra | 17:54 | |
| openstackgerrit | Doug Hellmann proposed openstack/requirements: Move data validation out of the unit tests and strengthen https://review.openstack.org/205290 | 17:55 |
| fungi | jeblair: mordred: clarkb: yolanda: skimming briefly, it seems that the pip module's init class is expecting a manage_pip_conf which defaults to false per 168309 but 166252 did not pass that in? | 17:55 |
| *** devkulkarni has quit IRC | 17:56 | |
| mordred | fungi: I was just about to say that same thing | 17:56 |
| *** e0ne has joined #openstack-infra | 17:56 | |
| mordred | fungi: and I believe that a patch that adds that param will fix it | 17:56 |
| fungi | mordred: right, i'm writing it now | 17:57 |
| mordred | I will approve it when you have written it | 17:57 |
| *** xyang1 has quit IRC | 17:58 | |
| openstackgerrit | Sirushti Murugesan proposed openstack-infra/project-config: Make the Heat grenade job voting https://review.openstack.org/201052 | 17:58 |
| openstackgerrit | Jeremy Stanley proposed openstack-infra/system-config: Manage pip in our server template class https://review.openstack.org/207149 | 18:00 |
| fungi | mordred: jeblair: clarkb: yolanda: ^ | 18:01 |
| *** jsavak has quit IRC | 18:02 | |
| fungi | alternatively, i can revert 166252 and then repropose it with that addition if preferred | 18:02 |
| jeblair | lgtm and images deleted | 18:03 |
| fungi | thanks | 18:03 |
| *** esmute_ has joined #openstack-infra | 18:03 | |
| *** prad has quit IRC | 18:04 | |
| greghaynes | fungi: oh, something I forgot to mention when reading meeting backlog - our image uploads to rax arent parellel across images but since our swift image uploads are multipart those parts happen in parallel behind the scenes of swiftclient | 18:05 |
| * nibalizer looks | 18:05 | |
| *** e0ne has quit IRC | 18:06 | |
| *** adam_g has joined #openstack-infra | 18:07 | |
| *** adam_g has quit IRC | 18:07 | |
| *** adam_g has joined #openstack-infra | 18:07 | |
| fungi | greghaynes: yeah, that got pointed out by clarkb | 18:07 |
| greghaynes | ah, ok | 18:07 |
| *** davidjc has quit IRC | 18:08 | |
| nibalizer | its already +a but 207149 lgtm | 18:08 |
| fungi | wonder if swiftclient has a mechanism to serialize multi-part uploads | 18:08 |
| fungi | nibalizer: thanks for the double-check | 18:08 |
| mordred | fungi: do we want to serialize it? | 18:09 |
| fungi | it was sort of a stab in the dark based on 30 seconds skimming the change | 18:09 |
| greghaynes | fungi: IIRC theres two ways to do uploads, one way doesnt support multipart and the other way doesnt let you control the behind the scenes threading | 18:09 |
| nibalizer | it's pretty straightforward | 18:09 |
| greghaynes | mordred: I think so, we at least want to be able to request rate limit | 18:09 |
| pabelanger | zuul having issues building images ATM? The inuse nodes looks pretty small right now | 18:09 |
| mordred | greghaynes, fungi: you can pass in options to configure the number of threads | 18:09 |
| *** e0ne has joined #openstack-infra | 18:09 | |
| greghaynes | oh! | 18:10 |
| mordred | segment_threads=self._options['segment_threads'], | 18:10 |
| mordred | object_dd_threads=self._options['object_dd_threads'], | 18:10 |
| greghaynes | nice | 18:10 |
| mordred | object_uu_threads=self._options['object_uu_threads'], | 18:10 |
| nibalizer | fungi: well kindof... | 18:10 |
| mordred | container_threads=self._options['container_threads'] | 18:10 |
| openstackgerrit | Matthew Treinish proposed openstack-infra/project-config: Add stable compat jobs to trove-integration repo https://review.openstack.org/207055 | 18:10 |
| fungi | mordred: well, if it helps us avoid overrunning our network connection (still not certain that's the issue but it's a possible explanation) then it might be something we want to tune | 18:10 |
| nibalizer | oh now I see it | 18:10 |
| pabelanger | nibalizer: would be interested in your feedback too: https://review.openstack.org/#/c/203164/6 | 18:10 |
| mtreinish | anteaya: ^^^ nm, I got it. I was just being dumb | 18:10 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config: Update constraints after releases and reqs changes https://review.openstack.org/202210 | 18:10 |
| clarkb | pabelanger see scrollback | 18:11 |
| fungi | pabelanger: yeah, jeblair just deleted today's images because 207149 is needed to get them working | 18:11 |
| nibalizer | pabelanger: the thing there that worries me is the template, does puppet undef become nil in a template? | 18:11 |
| yolanda | hi fungi, what's up? | 18:12 |
| fungi | yolanda: was just looking for confirmation that 207149 was the appropriate fix for our broken images today | 18:12 |
| yolanda | looking at that | 18:12 |
| pabelanger | clarkb: fungi thanks. I need better highlighting for zuul in irc | 18:13 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config: Rename .log to .log.txt in test-results publisher https://review.openstack.org/203099 | 18:13 |
| *** _nadya_ has quit IRC | 18:13 | |
| nibalizer | yolanda: 166252 landed which left new images without /etc/pip.conf | 18:13 |
| pabelanger | nibalizer: that syntax was take for existing merged puppet jobs | 18:13 |
| pabelanger | let me see if I can find something that was already merged | 18:13 |
| *** _nadya_ has joined #openstack-infra | 18:13 | |
| openstackgerrit | Merged openstack-infra/system-config: Use bifrost for bare metal portion of infra-cloud https://review.openstack.org/201666 | 18:14 |
| yolanda | fungi, so yes, that manage_pip_conf was set as an intention to avoid duplication errors on /etc/pip.conf | 18:15 |
| yolanda | manage_pip_conf shoudl be set to true, yes | 18:15 |
| *** AJaeger has joined #openstack-infra | 18:15 | |
| crinkle | nibalizer: pabelanger scope.lookupvar is derpy, this is what i had to do to get it to work https://review.openstack.org/#/c/203870/4/templates/git.vhost.erb | 18:15 |
| *** tiswanso has joined #openstack-infra | 18:16 | |
| yolanda | nibalizer, so we've broken stuff several days now, let's ensure we test and babysit the stuff before approval. I'll recheck all my queue to ensure that, because several changes are a bit old and not recently reviewed | 18:16 |
| nibalizer | so can we stop using scope.lookupvar? | 18:16 |
| yolanda | fungi, sorry about that, we need to improve it | 18:16 |
| *** FallenPegasus has joined #openstack-infra | 18:16 | |
| openstackgerrit | Merged openstack-infra/project-config: Add openstack/networking-calico project https://review.openstack.org/194709 | 18:16 |
| crinkle | nibalizer: no because there are a lot of places where the templates are looking stuff up in other classes | 18:17 |
| crinkle | i mean we could stop doing that but it wouldn't be trivial | 18:17 |
| clarkb | usually with vhost templates | 18:17 |
| crinkle | yeah | 18:17 |
| crinkle | it sort of makes sense there | 18:17 |
| nibalizer | yeah | 18:17 |
| clarkb | caller makes template that refers back to itself but its populated in httpd/apache context | 18:17 |
| *** _nadya_ has quit IRC | 18:18 | |
| *** dannywilson has joined #openstack-infra | 18:18 | |
| openstackgerrit | Merged openstack-infra/project-config: Add periodic kilo & juno jobs to murano https://review.openstack.org/201233 | 18:18 |
| openstackgerrit | litong01 proposed openstack-infra/project-config: Add new kiloeyes repository for OpenStack monitoring service https://review.openstack.org/203259 | 18:18 |
| nibalizer | what we can do is make the httpd::vhost able to take a content => template('something/myvhost.erb') | 18:18 |
| openstackgerrit | Merged openstack-infra/project-config: Remove CI for pecan. https://review.openstack.org/206635 | 18:18 |
| openstackgerrit | Merged openstack-infra/system-config: Manage pip in our server template class https://review.openstack.org/207149 | 18:18 |
| *** coolsvap is now known as coolsvap|brb | 18:18 | |
| nibalizer | then the template is evaluated in the context of the caller | 18:18 |
| openstackgerrit | Merged openstack-infra/project-config: Skip failing ceph backup test https://review.openstack.org/206759 | 18:18 |
| crinkle | nibalizer: that sounds good | 18:18 |
| clarkb | nibalizer how? isnt it by definition of being part of http in https context? | 18:19 |
| fungi | yolanda: no need to apologies. as they say, | 18:19 |
| fungi | you can't make an omelette without cracking the eggs | 18:19 |
| clarkb | puppet doesnt have closures aiui | 18:19 |
| mordred | clarkb: no - because as it is now, we pass in the template file, not the template context | 18:19 |
| *** jsavak has joined #openstack-infra | 18:19 | |
| nibalizer | clarkb: im saying you pass a 'content' string to httpd::vhost | 18:19 |
| nibalizer | and that content string is a vhost | 18:19 |
| crinkle | right | 18:19 |
| clarkb | ah gotcha | 18:19 |
| nibalizer | and we get it from template()... ok cool | 18:19 |
| nibalizer | but yea, nontrivial | 18:20 |
| *** prad has joined #openstack-infra | 18:20 | |
| ihrachyshka | mriedem, hi. I have some patches with +W that are not checked in gate though. is it related to failure today? should I reverify? | 18:20 |
| *** crc32 has joined #openstack-infra | 18:20 | |
| yolanda | anyway, i don't like breaking stuff, it's a bit more complicated in upstream where changes take so long to land, than downstream where we have the control, so i need to be better on that | 18:20 |
| nibalizer | also more better beaker testing should help with that | 18:21 |
| *** abregman has joined #openstack-infra | 18:21 | |
| mriedem | ihrachyshka: which? | 18:21 |
| mriedem | there was comment this morning that you might need to recheck things if they aren't in the zuul status page | 18:21 |
| openstackgerrit | Davide Agnello proposed openstack-infra/project-config: Adding RALLY_SCENARIO venv for Cue job https://review.openstack.org/201285 | 18:22 |
| yolanda | nibalizer, taking about beaker, i +2 all the changes, we had a conversation with Jim about that | 18:22 |
| AJaeger | ihrachyshka: yes, better: toggle the +W. If you recheck or reverify it will run bot hcheck and gate queue, if you toggle +W only gate | 18:22 |
| yolanda | we initially go with it | 18:22 |
| yolanda | i'll try adding an openstack-infra/cookiecutter | 18:22 |
| yolanda | and let's iterate in the future to have it reusable | 18:23 |
| ihrachyshka | AJaeger, thanks! btw that hook for qos worked :) | 18:23 |
| *** jasondotstar has quit IRC | 18:23 | |
| AJaeger | ihrachyshka: Party time! ;) | 18:23 |
| ihrachyshka | woohoo | 18:23 |
| yolanda | btw, jeblair https://review.openstack.org/#/c/207109/ - do i need governance path? | 18:23 |
| yolanda | patch? | 18:23 |
| openstackgerrit | David Shrewsbury proposed openstack-infra/shade: Be smarter finding private IP https://review.openstack.org/207157 | 18:23 |
| *** dims_ has joined #openstack-infra | 18:24 | |
| clarkb | yolanda why do we need a different cookiecutter? | 18:24 |
| *** prad has quit IRC | 18:24 | |
| openstackgerrit | David Shrewsbury proposed openstack-infra/shade: Be smarter finding private IP https://review.openstack.org/207157 | 18:25 |
| Shrews | mordred: rcarrillocruz: ^^^^ | 18:25 |
| jeblair | yolanda: yes, to add it to the infra project. you can make the governance patch Depends-On this one (you don't have to do it first) | 18:25 |
| yolanda | jeblair ok | 18:25 |
| mrmartin | fungi: re, the upstream solr-vamsee puppet module required to solve the ask puppet failure of smartctn have a new release 0.0.8 including my fantastic pull request. So we have this patch in openstack-infra/system-config: https://review.openstack.org/#/c/205768/ with a wunderschön support from infra members, can I get an approve on that? :) | 18:25 |
| fungi | AJaeger: ihrachyshka: if it has no verified vote from jenkins, recheck because it has go back to check before gate regardless (and i believe workflow +1 votes will be ignored in that situation), but if it has a verified +1 from jenkins then recheck does no good--you actually _need_ to add/toggle workflow +1 | 18:25 |
| mrmartin | I hope I'm not breaking eggs :) | 18:26 |
| clarkb | isnt it generaly the same? metadata.json, gemfile, rakefile, spec/ | 18:26 |
| *** annegentle has quit IRC | 18:26 | |
| ihrachyshka | fungi, I toggled and all four got into gate. awesome! | 18:26 |
| ihrachyshka | thank you all | 18:26 |
| yolanda | clarkb, mostly to reference all openstack-infra bits, samples etc | 18:26 |
| *** pal has quit IRC | 18:26 | |
| yolanda | openstack cookiecutter references incorrect repos for example | 18:26 |
| *** annegentle has joined #openstack-infra | 18:26 | |
| fungi | mrmartin: great--thanks for working on that! | 18:26 |
| yolanda | git clone https://review.openstack.org/openstack/puppet-$proj | 18:26 |
| yolanda | and has different examples | 18:26 |
| mrmartin | fungi: thnx | 18:26 |
| *** prad has joined #openstack-infra | 18:27 | |
| clarkb | you have to edit it regardless right? | 18:27 |
| openstackgerrit | Merged openstack-infra/shade: Document create_object https://review.openstack.org/206872 | 18:27 |
| Shrews | mordred: i tested that patch on hp, rax, and dreamhost. all good on those | 18:27 |
| clarkb | Shrews no devstack? | 18:27 |
| Shrews | bah | 18:28 |
| *** dims has quit IRC | 18:28 | |
| clarkb | I am sure its fine :) | 18:28 |
| *** ayoung has quit IRC | 18:29 | |
| Shrews | clarkb: you didn't know that all of those providers run devstack for their clouds??? :) | 18:29 |
| mordred | Shrews: woot | 18:29 |
| mordred | looks great | 18:29 |
| *** amickus has joined #openstack-infra | 18:30 | |
| marcusvrn | hi all!! Is there any third party coordinator online? | 18:31 |
| *** openstackgerrit has quit IRC | 18:31 | |
| *** openstackgerrit has joined #openstack-infra | 18:32 | |
| *** armax has quit IRC | 18:32 | |
| *** jsavak has quit IRC | 18:33 | |
| *** jsavak has joined #openstack-infra | 18:34 | |
| *** dannywilson has quit IRC | 18:34 | |
| *** jaypipes has joined #openstack-infra | 18:34 | |
| *** dannywilson has joined #openstack-infra | 18:34 | |
| AJaeger | yolanda: the governance patch is pure housekeeping in this case | 18:34 |
| yolanda | i always forget about it | 18:35 |
| rcarrillocruz | Shrews: happy to see that :-), my playbooks relying on private IPs from inventory were broken | 18:36 |
| *** thorst has quit IRC | 18:36 | |
| rcarrillocruz | Shrews: so, you also have account on dreamhost? | 18:36 |
| rcarrillocruz | i'm jealous | 18:36 |
| Shrews | rcarrillocruz: that should fix your playbooks. please test that patch to make sure. | 18:37 |
| rcarrillocruz | will do thx | 18:37 |
| Shrews | rcarrillocruz: yes, reed was nice enough to set me up a dreamhost account. turns out, providers like for shade to work with them :) | 18:38 |
| rcarrillocruz | hah | 18:38 |
| *** shashank_hegde has quit IRC | 18:38 | |
| rcarrillocruz | yeah, kind of hard to write patches without visibility on how the various providers implement things :/ | 18:39 |
| Shrews | s/kind of/VERY/ :) | 18:39 |
| fungi | hopefully those providers don't take shade's existence as an excuse to avoid making an effort toward interoperability with other providers | 18:39 |
| Shrews | fungi: it is a bug that shade exists. mordred has said such in his talks, iirc | 18:40 |
| *** salv-orlando has quit IRC | 18:40 | |
| openstackgerrit | Merged openstack-infra/system-config: Update vamsee-solr to 0.0.8 https://review.openstack.org/205768 | 18:40 |
| fungi | yep, but those providers may not be going to mordred's talks or witnessing our discussions/proclamations (as much as i wish they did) | 18:40 |
| rcarrillocruz | hah, indeed | 18:40 |
| Shrews | fungi: they miss mordred's talks at their own peril! | 18:41 |
| fungi | that they did... that they did | 18:41 |
| Shrews | mordred: i have written your first slide on all of your future talks ^^^^^ | 18:41 |
| dhellmann | hey, folks, I'm seeing some errors from etherpad. Trying to load https://etherpad.openstack.org/p/havana-python3 gives me the error in http://paste.openstack.org/show/406298/ -- has anyone seen that before? | 18:42 |
| *** vinay_yadhav has quit IRC | 18:42 | |
| *** bhuvan has joined #openstack-infra | 18:43 | |
| fungi | dhellmann: weird, i can load that one just fine | 18:43 |
| dhellmann | fungi: hmm, maybe it's a browser issue, let me try again | 18:43 |
| dhellmann | I did reload 2 times and see the same message | 18:43 |
| fungi | this is with firefox (well, iceweasel) 38 | 18:43 |
| *** sdake has quit IRC | 18:43 | |
| dhellmann | yeah, worked well with firefox for me, too | 18:44 |
| openstackgerrit | Julia Kreger proposed openstack-infra/glean: Create /root/.ssh folder if folder does not exist https://review.openstack.org/207166 | 18:44 |
| dhellmann | maybe I need to restart safari | 18:44 |
| dhellmann | ok, ignore me, I'll come back if I continue to have trouble | 18:44 |
| EmilienM | dhellmann: works for me (fedora/firefox) | 18:44 |
| mordred | Shrews: +100 | 18:45 |
| * TheJulia grumbles at glean, and goes back to hide under her rock | 18:45 | |
| *** bharath_ has joined #openstack-infra | 18:45 | |
| dhellmann | EmilienM, fungi: restarting the browser fixed it. weird. | 18:45 |
| *** fawadkhaliq has joined #openstack-infra | 18:45 | |
| *** sdake has joined #openstack-infra | 18:46 | |
| *** soren has joined #openstack-infra | 18:46 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/shade: Start using keystoneauth for keystone sessions https://review.openstack.org/183176 | 18:48 |
| mordred | morganfainberg: ^^ | 18:48 |
| openstackgerrit | Merged openstack-infra/release-tools: Support deliverables in milestone.sh https://review.openstack.org/206523 | 18:50 |
| rcarrillocruz | TheJulia: i hear ya | 18:51 |
| rcarrillocruz | :D | 18:51 |
| clarkb | can always use cloud init | 18:51 |
| *** thorst has joined #openstack-infra | 18:51 | |
| clarkb | its not bad until you need it to not mount ephemeral drives | 18:51 |
| clarkb | or create an ubuntu user | 18:51 |
| clarkb | or fail against metadata service | 18:52 |
| Shrews | mordred: https://review.openstack.org/191765 seems to break neutron | 18:55 |
| *** Sukhdev_ has joined #openstack-infra | 18:55 | |
| *** fawadk has joined #openstack-infra | 18:55 | |
| *** fawadkhaliq has quit IRC | 18:56 | |
| Shrews | mordred: neutron_client used to use auth_token, but not anymore with that change | 18:56 |
| Shrews | so maybe that is thing | 18:57 |
| *** pal has joined #openstack-infra | 18:57 | |
| pleia2 | I'm helping the translations folks move over from #openstack-translation to #openstack-i18n this week, if I could get some eyes on this easy change I'd appreciate it: https://review.openstack.org/#/c/205979/ | 19:00 |
| timrc | dhellmann: Loaded for me as well in firefox. | 19:00 |
| pleia2 | I'm setting up the redirect for them on Friday, late in the day we can also approve https://review.openstack.org/#/c/205980/ (so a +2 would be helpful so I can +A when it's no-meeting go time) | 19:01 |
| nikeshm__ | https://review.openstack.org/#/c/203727/ still not merged | 19:01 |
| *** salv-orlando has joined #openstack-infra | 19:01 | |
| *** sergmelikyan has quit IRC | 19:03 | |
| *** sputnik13 has quit IRC | 19:03 | |
| *** krtaylor has quit IRC | 19:04 | |
| jeblair | (apropos of nothing, openstack-tra<tab> is easier for me to type than openstack-i18<tab>) | 19:04 |
| jeblair | pleia2: +2 | 19:06 |
| pleia2 | jeblair: thanks, sir! | 19:06 |
| AJaeger | pleia2: do you want to WIP https://review.openstack.org/205980 in this case? | 19:06 |
| AJaeger | pleia2: I meant the parent: https://review.openstack.org/#/c/205979/ ? | 19:07 |
| pleia2 | AJaeger: yeah, I'll do that | 19:07 |
| *** flepied1 has joined #openstack-infra | 19:09 | |
| *** soren has quit IRC | 19:09 | |
| *** coolsvap|brb is now known as coolsvap|away | 19:10 | |
| *** vinay_yadhav has joined #openstack-infra | 19:10 | |
| *** sergmelikyan has joined #openstack-infra | 19:11 | |
| dhellmann | fungi, jeblair, clarkb, pleia2 : how long after seeing a "success" message at the end of the pre-release tarball job is it reasonable to need to wait for the job to "finish"? https://jenkins02.openstack.org/job/neutron-tarball/6/consoleFull | 19:12 |
| *** shashank_hegde has joined #openstack-infra | 19:12 | |
| *** flepied has quit IRC | 19:12 | |
| *** geoffarnold has joined #openstack-infra | 19:12 | |
| *** ZZelle has joined #openstack-infra | 19:13 | |
| dhellmann | I'm trying to figure out if that's been hung for ~15 minutes, or if it's just slow because of load or something | 19:13 |
| *** cburroughs has joined #openstack-infra | 19:13 | |
| *** dzamboni has quit IRC | 19:13 | |
| jeblair | dhellmann: why do you say it is not finished? (i may be missing something) | 19:14 |
| *** jsavak has quit IRC | 19:14 | |
| pleia2 | dhellmann: we have a long post queue, dig around on http://status.openstack.org/zuul/ to look | 19:14 |
| Shrews | jeblair: instead of <tab> on the new one, just type the final letter 'n' :-P | 19:14 |
| nikeshm__ | jeblair : https://review.openstack.org/#/c/203727/ new patch? | 19:14 |
| dhellmann | jeblair: well, jenkins is reporting it as done, but it's still appearing on the zuul list and -- well, it just cleared | 19:14 |
| morganfainberg | mordred: scary | 19:14 |
| jeblair | Shrews: can i just ask you to do it for me if you think it's so easy! ;) | 19:14 |
| morganfainberg | mordred: it shouldnt change tooooooo much after this | 19:14 |
| *** jsavak has joined #openstack-infra | 19:15 | |
| *** devkulkarni has joined #openstack-infra | 19:15 | |
| Shrews | jeblair: absolutely! ask away heeh | 19:15 |
| Shrews | hehe... typing *is* hard | 19:15 |
| dhellmann | pleia2: do those run after tags sometimes? I thought those were post-merge jobs? | 19:15 |
| *** teran_ has joined #openstack-infra | 19:15 | |
| *** fawadkhaliq has joined #openstack-infra | 19:16 | |
| jeblair | dhellmann: zuul is just extremely busy setting the same jenkins build descriptions over and over. https://review.openstack.org/195785 will improve the situation. | 19:16 |
| dhellmann | jeblair: ah, ok. I know we had issues earlier today, so between that job appearing to hang and the next release tag job not showing up for a while (it just did) I was worried I was the canary for another issue. | 19:17 |
| *** krtaylor has joined #openstack-infra | 19:17 | |
| * dhellmann takes a deep breath, tries to be more patient, and makes a note to not tag releases late in the day when the queues are full | 19:17 | |
| jeblair | a solved one, this time, fortunately | 19:18 |
| dhellmann | cool | 19:18 |
| jeblair | dhellmann: or we could just fix zuul so you don't have to change your behavior :) | 19:18 |
| *** esmute_ has quit IRC | 19:18 | |
| dhellmann | jeblair: I'm trying to not be *too* demanding. ;-) | 19:18 |
| *** jasondotstar has joined #openstack-infra | 19:18 | |
| *** teran has quit IRC | 19:19 | |
| *** fawadk has quit IRC | 19:19 | |
| *** tjones has quit IRC | 19:21 | |
| *** vinay_yadhav has quit IRC | 19:23 | |
| clarkb | anteaya: I am looking for that revert of the mulitnode msoke jobs, am I remembering correctly that you pushed a change for that? | 19:23 |
| *** Apsu has quit IRC | 19:24 | |
| clarkb | anteaya: I need to rebaes it atop a change to test ubuntu-trusty nodes using the full test | 19:24 |
| *** yamamoto has quit IRC | 19:24 | |
| *** fawadk has joined #openstack-infra | 19:24 | |
| *** Apsu has joined #openstack-infra | 19:24 | |
| openstackgerrit | Clark Boylan proposed openstack-infra/project-config: Run full multinode tests against new dib images https://review.openstack.org/207187 | 19:25 |
| clarkb | mordred: greghaynes ^ that should give us a bit mor econfidence in those images | 19:25 |
| openstackgerrit | Merged openstack-infra/project-config: dragonflow: Enable python3 jobs https://review.openstack.org/207093 | 19:26 |
| *** fawadkhaliq has quit IRC | 19:26 | |
| *** dims_ has quit IRC | 19:27 | |
| *** davideagnello has quit IRC | 19:27 | |
| *** vinay_yadhav has joined #openstack-infra | 19:27 | |
| *** dims has joined #openstack-infra | 19:27 | |
| *** esmute_ has joined #openstack-infra | 19:27 | |
| *** devkulkarni has quit IRC | 19:30 | |
| *** jsavak has quit IRC | 19:30 | |
| greghaynes | clarkb: yea, although last I checked they were failing anyways due to not enough disk space because were not resizing | 19:31 |
| *** esmute_ has quit IRC | 19:31 | |
| *** jsavak has joined #openstack-infra | 19:33 | |
| dhellmann | jeblair: is there a log of recent jobs run? I'm trying to look at the pre-release tag job log for neutron-lbaas 7.0.0.0b2 to understand why the tarball isn't there, and the URL that git-os-job produces gives a 404 http://logs.openstack.org/db/db9f09a1887b8c84811497b8571e71db219f1e22/ | 19:34 |
| jeblair | dhellmann: in that case we have to hit all of the jenkinses; what's the job name? | 19:34 |
| dhellmann | I don't know off the top of my head, the one that handles pre-release tagging? Let me see if I can figure that out. | 19:35 |
| clarkb | greghaynes: ok, at least this way we are ready for the next issue once thats fixed | 19:35 |
| openstackgerrit | Clark Boylan proposed openstack-infra/project-config: Switch to using full multinode jobs everywhere https://review.openstack.org/207191 | 19:35 |
| greghaynes | clarkb: yerp | 19:35 |
| *** salv-orlando has quit IRC | 19:35 | |
| clarkb | anteaya: ^ I went ahead and di that which isn't quite a revert | 19:35 |
| dhellmann | jeblair: neutron-lbaas-tarball? | 19:35 |
| jeblair | dhellmann: yeah, so maybe: https://jenkins05.openstack.org/job/neutron-lbaas-tarball/3/console | 19:35 |
| dhellmann | jeblair: the time looks about right, yeah | 19:36 |
| AJaeger | dhellmann, jeblair: Might that be a job that was in the post queue while zuul got restarted? I don't think that we enqueue post and periodic jobs again... | 19:36 |
| dhellmann | AJaeger: zuul-cloner died, according to the log ^^ | 19:37 |
| *** vinay_yadhav has quit IRC | 19:37 | |
| jeblair | ah, and it did make it to swift: http://logs.openstack.org/db/db9f09a1887b8c84811497b8571e71db219f1e22/pre-release/neutron-lbaas-tarball/ba494f7 | 19:37 |
| jeblair | all you need to find that is to guess the correct random number at the end | 19:38 |
| jeblair | (this is a known issue that will be fixed before we roll out swift logs to more jobs) | 19:38 |
| clarkb | we can also switch post jobs back to the normal apache setup | 19:38 |
| clarkb | until that is fixed | 19:38 |
| jeblair | might be worth doing | 19:38 |
| clarkb | I will write that change now | 19:39 |
| jeblair | dhellmann, clarkb: so.... creating the venv failed | 19:40 |
| * dhellmann tries it locally | 19:41 | |
| dhellmann | jeblair: random numbers make things more interesting :-) | 19:41 |
| jeblair | dhellmann, clarkb: because it runs tools/tox_install.sh to install neutron | 19:41 |
| dhellmann | guh | 19:42 |
| jeblair | dhellmann: i suspect it will work locally unless you are running on a test node | 19:42 |
| dhellmann | no, I'm just using a standard cloud box | 19:42 |
| jeblair | it appears to make the assumption that if you are running on an infra CI system then you must be running to test a proposed change | 19:42 |
| *** tjones has joined #openstack-infra | 19:42 | |
| jeblair | clarkb: but you and i have been working to make zuul-cloner require fewer parameters and fall back correctly | 19:43 |
| jeblair | i don't know whether those changes have made it onto the workers yet though | 19:43 |
| dhellmann | mestery: we hit a snag with neutron-lbaas ^^ | 19:43 |
| openstackgerrit | Merged openstack-infra/project-config: Create a repository for stackforge/xstatic-bootswatch https://review.openstack.org/206732 | 19:44 |
| dhellmann | jeblair: you're right that it worked locally. | 19:44 |
| openstackgerrit | Clark Boylan proposed openstack-infra/project-config: Write post job logs to apache and swift https://review.openstack.org/207195 | 19:45 |
| dhellmann | jeblair: have any suggestions for what to do with this? We have the tag on the repo now, but no tarball. :-/ | 19:45 |
| clarkb | jeblair: dhellmann ^ that change will make it easier to find the logs | 19:45 |
| jeblair | i'm looking to see if the current version of zuul-cloner deals with this | 19:45 |
| * mestery reads backscroll | 19:46 | |
| *** pal has quit IRC | 19:46 | |
| *** achanda has quit IRC | 19:46 | |
| clarkb | jeblair: this sound slike a case of far too intelligent tox and (mis|un)communicated needs | 19:46 |
| clarkb | ? | 19:46 |
| *** krtaylor has quit IRC | 19:46 | |
| openstackgerrit | Merged openstack-infra/project-config: Add fuel-mirror repo https://review.openstack.org/205963 | 19:46 |
| jeblair | clarkb: no i think we said "if you need to install neutron as a dep, you should use zuul-cloner" | 19:47 |
| clarkb | oh this is that | 19:47 |
| clarkb | ya I remember saying that | 19:47 |
| *** sergmelikyan has quit IRC | 19:47 | |
| mestery | clarkb jeblair: I remember these patches from dougwig a while back around the zuul cloner stuff ... but only slightly. | 19:47 |
| mestery | Is that what's causing the release issues dhellmann pinged about? | 19:48 |
| dhellmann | mestery: yeah, the tox env failed to build, so the job to build the tarball failed | 19:48 |
| dhellmann | mestery: http://logs.openstack.org/db/db9f09a1887b8c84811497b8571e71db219f1e22/pre-release/neutron-lbaas-tarball/ba494f7/console.html | 19:48 |
| mestery | dhellmann: #sadpanda | 19:48 |
| *** HenryG has quit IRC | 19:49 | |
| *** pal has joined #openstack-infra | 19:49 | |
| mestery | yikes | 19:49 |
| jeblair | clarkb: https://review.openstack.org/195184 would fix it | 19:50 |
| * mestery dhellmann: I think this is the commit which introduced the zuul cloner stuff: https://review.openstack.org/#/c/188626/ | 19:50 | |
| *** otter768 has joined #openstack-infra | 19:50 | |
| *** sputnik13 has joined #openstack-infra | 19:50 | |
| mestery | jeblair: Am I about 10 steps behind you and clarkb now, or are you talkign about a different issue? :) | 19:50 |
| jeblair | mestery: same issue -- | 19:50 |
| mestery | jeblair: Ack | 19:50 |
| *** e0ne has quit IRC | 19:51 | |
| jeblair | mestery: we have been enhancing zuul-cloner to support this use-case better | 19:51 |
| mestery | jeblair: Cool, thanks! I know Doug was on the cutting edge with this stuff when he moved forward with it :) | 19:51 |
| *** salv-orlando has joined #openstack-infra | 19:51 | |
| jeblair | mestery: i ran zuul-cloner with the change in https://review.openstack.org/195184 applied and in the environment from that job, and it did the right thing | 19:51 |
| jeblair | clarkb: ^ | 19:51 |
| *** blogan has joined #openstack-infra | 19:51 | |
| mestery | jeblair: Awesome that you already have a fix posted for review :) | 19:52 |
| jeblair | so the approach is not fundamentally flawed, though it will take a while for that change to make it onto the workers | 19:52 |
| jeblair | i will rebase that on other changes which should fix the test failures | 19:52 |
| *** HenryG has joined #openstack-infra | 19:52 | |
| dhellmann | jeblair: at that point could we retrigger the tarball job? | 19:52 |
| *** glauco has quit IRC | 19:53 | |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul: cloner: make more zuul args optional https://review.openstack.org/195184 | 19:53 |
| jeblair | rebased ^ | 19:53 |
| jeblair | dhellmann: yes. so what kind of timeframe are you on? | 19:53 |
| dhellmann | jeblair: this is the L2 milestone tag, so as long as we get it done some time thursday I think we'll still be on schedule | 19:54 |
| *** otter768 has quit IRC | 19:54 | |
| dhellmann | mestery: I'm going to hold off tagging anything else for neutron until this is resolved | 19:54 |
| clarkb | will rereview | 19:54 |
| mestery | dhellmann: Ack on that | 19:55 |
| *** FallenPegasus is now known as MarkAtwood | 19:55 | |
| mestery | clarkb jeblair: Thanks for the help here! | 19:55 |
| *** e0ne has joined #openstack-infra | 19:55 | |
| *** tqtran has joined #openstack-infra | 19:55 | |
| *** Sherif has joined #openstack-infra | 19:55 | |
| blogan | jeblair: is that fix going into neutron-lbaas? | 19:55 |
| jeblair | blogan: no, just zuul | 19:55 |
| blogan | got in this late | 19:55 |
| blogan | okay, thanks | 19:56 |
| *** mrmartin has quit IRC | 19:56 | |
| openstackgerrit | Merged openstack-infra/devstack-gate: Add a loop to git_prune https://review.openstack.org/183124 | 19:56 |
| dhellmann | clarkb, jeblair : ++ thanks for the quick fix | 19:56 |
| *** nikeshm__ has quit IRC | 19:56 | |
| jeblair | clarkb, dhellmann: evil idea to speed this through: add 'rm -rf /usr/zuul-env/bin/zuul-cloner' to the tarball jobs in jjb (right before running the script) | 19:57 |
| jeblair | just until the zuul-cloner changes land and we have new images | 19:57 |
| *** crc32 has quit IRC | 19:57 | |
| dhellmann | jeblair: that sounds naughty | 19:58 |
| *** ajmiller has quit IRC | 19:58 | |
| *** ajmiller has joined #openstack-infra | 19:59 | |
| *** krtaylor has joined #openstack-infra | 19:59 | |
| *** koolhead17 has joined #openstack-infra | 20:00 | |
| clarkb | I am fine with that | 20:01 |
| *** sergmelikyan has joined #openstack-infra | 20:01 | |
| openstackgerrit | James E. Blair proposed openstack-infra/project-config: Temporarily remove zuul-cloner before tarball https://review.openstack.org/207200 | 20:01 |
| openstackgerrit | James E. Blair proposed openstack-infra/project-config: Revert "Temporarily remove zuul-cloner before tarball" https://review.openstack.org/207201 | 20:01 |
| jeblair | clarkb, dhellmann: ^ | 20:02 |
| *** fawadk has quit IRC | 20:02 | |
| *** puranamr has quit IRC | 20:02 | |
| openstackgerrit | James E. Blair proposed openstack-infra/project-config: Revert "Temporarily remove zuul-cloner before tarball" https://review.openstack.org/207201 | 20:02 |
| * jeblair remembers depends-on | 20:02 | |
| *** e0ne has quit IRC | 20:03 | |
| jeblair | oh, 200 needs a revision | 20:03 |
| *** hodos has joined #openstack-infra | 20:03 | |
| *** puranamr has joined #openstack-infra | 20:03 | |
| *** shashank_hegde has quit IRC | 20:03 | |
| openstackgerrit | Merged openstack/requirements: Updated from generate-constraints https://review.openstack.org/206650 | 20:04 |
| openstackgerrit | James E. Blair proposed openstack-infra/project-config: Revert "Temporarily remove zuul-cloner before tarball" https://review.openstack.org/207201 | 20:05 |
| openstackgerrit | James E. Blair proposed openstack-infra/project-config: Temporarily remove zuul-cloner before tarball https://review.openstack.org/207200 | 20:05 |
| jeblair | dhellmann, clarkb: sorry ^ | 20:05 |
| *** crc32 has joined #openstack-infra | 20:05 | |
| *** shashank_hegde has joined #openstack-infra | 20:06 | |
| dhellmann | jeblair: +1 | 20:06 |
| *** tiswanso has quit IRC | 20:06 | |
| jeblair | dhellmann, mestery: this will be affecting the branch-tip tarballs too... you can see they haven't been updated since 9 july: http://tarballs.openstack.org/neutron-vpnaas/ | 20:08 |
| jeblair | [ ]neutron-vpnaas-master.tar.gz09-Jul-2015 02:30 135K | 20:08 |
| mestery | jeblair: Yes :( | 20:08 |
| jeblair | dhellmann: and you can use that to predict whether this will fail for other repos | 20:08 |
| *** AJaeger has quit IRC | 20:09 | |
| dhellmann | jeblair: I don't understand enough about this to know why it doesn't cause the regular unit test jobs to fail to build their virtualenvs. Is that obvious to you? | 20:09 |
| dhellmann | jeblair, mestery : interestingly, fwaas appears ok | 20:10 |
| *** e0ne has joined #openstack-infra | 20:11 | |
| mestery | dhellmann jeblair: fwaas is ok but vpnass failed? Weird. | 20:12 |
| *** baoli_ has quit IRC | 20:12 | |
| dhellmann | mestery: cosmic rays? | 20:13 |
| mestery | lol | 20:13 |
| jeblair | dhellmann: yes -- zuul provides different env variables based on the context it's running a job in. jobs that test proposed changes have one set; periodic jobs have another set; and post (including tag) jobs have a third set | 20:14 |
| *** koolhead17 has quit IRC | 20:14 | |
| dhellmann | jeblair: so zuul-cloner is getting different inputs in those other cases that allow it to work, but in the tarball job it is missing something? | 20:15 |
| jeblair | yep, and we haven't used it in a tarball job yet so haven't seen this until now | 20:16 |
| dhellmann | makes sense | 20:16 |
| *** openstackgerrit has quit IRC | 20:16 | |
| *** openstackgerrit has joined #openstack-infra | 20:17 | |
| *** tqtran has quit IRC | 20:17 | |
| *** annegentle has quit IRC | 20:17 | |
| clarkb | finishing up lunch then will do some reviews | 20:17 |
| *** jasondotstar has quit IRC | 20:19 | |
| *** rkukura_ has joined #openstack-infra | 20:23 | |
| *** yamamoto has joined #openstack-infra | 20:24 | |
| *** achanda has joined #openstack-infra | 20:25 | |
| *** rkukura has quit IRC | 20:26 | |
| *** rkukura_ is now known as rkukura | 20:26 | |
| *** xyang1 has joined #openstack-infra | 20:26 | |
| *** shashank_hegde has quit IRC | 20:28 | |
| *** xyang1 has quit IRC | 20:29 | |
| *** kaisers has left #openstack-infra | 20:29 | |
| *** yamamoto has quit IRC | 20:29 | |
| *** jasondotstar has joined #openstack-infra | 20:32 | |
| *** tiswanso has joined #openstack-infra | 20:32 | |
| *** tiswanso has quit IRC | 20:33 | |
| *** tiswanso has joined #openstack-infra | 20:33 | |
| *** tiswanso has quit IRC | 20:34 | |
| *** achanda has quit IRC | 20:34 | |
| *** achanda has joined #openstack-infra | 20:35 | |
| *** salv-orlando has quit IRC | 20:36 | |
| *** armax has joined #openstack-infra | 20:39 | |
| *** armax has quit IRC | 20:39 | |
| *** achanda has quit IRC | 20:41 | |
| *** davideagnello has joined #openstack-infra | 20:41 | |
| *** MarkAtwood has quit IRC | 20:41 | |
| *** MarkAtwood has joined #openstack-infra | 20:42 | |
| lifeless | fungi: so do we have the constraint job running on-demand on requirements landings? | 20:43 |
| *** kgiusti has left #openstack-infra | 20:43 | |
| lifeless | fungi: and on releases? | 20:43 |
| *** devkulkarni has joined #openstack-infra | 20:43 | |
| lifeless | separately, I think the proposal bot is incorrectly uploading new revisions when the diff hasn't changed. | 20:44 |
| lifeless | https://review.openstack.org/#/c/206825/ revs 2 and 3 are identical | 20:45 |
| lifeless | or - do I misunderstand the intent there? | 20:45 |
| *** sergmelikyan has quit IRC | 20:46 | |
| *** salv-orlando has joined #openstack-infra | 20:46 | |
| clarkb | the diff is non zero, if chceking a diff in the script that would be the problem | 20:46 |
| lifeless | I'm not sure, I thought I remembered you making it only propose when the diff had itself updated | 20:47 |
| clarkb | I don't think I did anything with that job so not sue how it is set up | 20:47 |
| *** shashank_hegde has joined #openstack-infra | 20:48 | |
| *** esmute_ has joined #openstack-infra | 20:48 | |
| *** puranamr has quit IRC | 20:48 | |
| *** tjones has quit IRC | 20:49 | |
| *** bswartz has quit IRC | 20:49 | |
| *** cdent has quit IRC | 20:50 | |
| *** tonytan4ever has quit IRC | 20:51 | |
| notnownikki | looks like a rebase to me, different parents, so perhaps ignore the parents in the commit message part? | 20:51 |
| lifeless | clarkb: oh, huh. my memory must be faulty :) | 20:52 |
| clarkb | yes thats the issue what it should be doing is diffing HEAD^ but I think its diffing against the old chnage | 20:52 |
| lifeless | clarkb: yes, thats what I think too | 20:53 |
| *** soren has joined #openstack-infra | 20:53 | |
| *** puranamr has joined #openstack-infra | 20:53 | |
| notnownikki | make sense to me, for what is worth :) | 20:53 |
| *** glauco has joined #openstack-infra | 20:53 | |
| ybathia | clarkb: Is there a bug tracking the stable/juno horizon failure? | 20:53 |
| *** pal has quit IRC | 20:54 | |
| clarkb | ybathia: I am not sure | 20:54 |
| ybathia | clarkb: ok, np, thanks | 20:55 |
| *** Sukhdev_ has quit IRC | 20:55 | |
| *** sergmelikyan has joined #openstack-infra | 20:55 | |
| clarkb | jeblair: on 195184 I don't think you need the new if self.zuul_branch check as zuul_brnach is either "" or "something" | 20:55 |
| notnownikki | would we still want to notify of a new proposal if the commit message was changed? | 20:55 |
| notnownikki | and nothing else, that is | 20:56 |
| clarkb | notnownikki: the commit message wont change as its constant | 20:56 |
| jeblair | clarkb: yeah, but the else sets it to None which is different than an re.sub on "" so will affect later behavior | 20:57 |
| clarkb | oh gotcha | 20:57 |
| jeblair | clarkb: (note the "if ... fallback_zuul_ref ..." on 160) | 20:57 |
| notnownikki | commit messages can't change? huh | 20:58 |
| clarkb | yup thanks | 20:58 |
| *** dimtruck is now known as zz_dimtruck | 20:58 | |
| clarkb | notnownikki: no its a constant bot generates commit | 20:58 |
| *** glauco has quit IRC | 20:58 | |
| notnownikki | oh! sorry, came in halfway through :) bot generated, makes sense now | 20:59 |
| clarkb | and a commit mesage only cange isn't useful there, we can just pick up a new constant message with the next actual changes | 20:59 |
| *** tonytan4ever has joined #openstack-infra | 20:59 | |
| openstackgerrit | Merged openstack-infra/project-config: Temporarily remove zuul-cloner before tarball https://review.openstack.org/207200 | 20:59 |
| clarkb | jeblair: ok that whole stack lgtm | 20:59 |
| *** crc32 has quit IRC | 21:01 | |
| clarkb | jeblair: I figure its ok to push them through without a second +2 if you like, diffs are small. Though jhesketh should be awake soon | 21:02 |
| jeblair | oh and look the job change merged, so hopefully later this afternoon we can retrigger it | 21:04 |
| ZZelle | mordred, hi | 21:05 |
| mordred | ZZelle: hi! I'm only partially here ... | 21:05 |
| *** Hal has joined #openstack-infra | 21:06 | |
| ZZelle | mordred, could you relook at your change https://review.openstack.org/109869 ? | 21:06 |
| *** glauco_ has joined #openstack-infra | 21:07 | |
| *** Hal is now known as Guest29701 | 21:07 | |
| ZZelle | mordred, it's becoming a mess to develop and rebase code in git-review :s | 21:07 |
| timrc | Clint: Hi. You seem to be in the know about the :httpd::logrotate class that yolanda has proposed. It seems like this class will override logrotate settings for all ${apache_logdir}/*.log -- but does that mean that we can not control log rotatio nfor specific types of logs (e.g. debug vs. error vs access)? | 21:09 |
| mordred | yes! it looks great ... let me pull it and test as soon as my plane takes off | 21:09 |
| fungi | lifeless: to answer your question, not yet https://review.openstack.org/202210 | 21:09 |
| reed | does anybody here have a Ubuntu 15.04 image for KVM? | 21:10 |
| fungi | i'm waiting to see if other reviewers second ianw's comment about wanting the job to continuously poll the mirrors until it sees them update | 21:10 |
| reed | weirdly related to openstack request ^^ :) | 21:10 |
| mordred | fungi: I think the change ZZelle linked ^^ is ready for you | 21:10 |
| Clint | timrc: with current design, no | 21:10 |
| openstackgerrit | Mohammad Banikazemi proposed openstack-infra/irc-meetings: Add Kuryr weekly IRC meeting https://review.openstack.org/206324 | 21:11 |
| *** ajo_ has joined #openstack-infra | 21:11 | |
| *** ajo_ has quit IRC | 21:12 | |
| *** yamamoto has joined #openstack-infra | 21:12 | |
| timrc | Clint: Hrm. | 21:12 |
| *** ajo_ has joined #openstack-infra | 21:12 | |
| *** ajo_ has quit IRC | 21:13 | |
| clarkb | fungi: make the job run in the release queue after successful upload | 21:13 |
| clarkb | fungi: then you don't have to worry or poll | 21:13 |
| * clarkb reads change to see if that is what is happening | 21:14 | |
| clarkb | ya that is what happens so not sure what the concern is | 21:14 |
| *** aysyd has quit IRC | 21:14 | |
| *** yamamoto has quit IRC | 21:14 | |
| fungi | clarkb: there's a race | 21:15 |
| clarkb | fungi: only if we use the mirrors there | 21:15 |
| clarkb | whih we can not do | 21:15 |
| fungi | clarkb: hence i have it sleep 5 minutes to give bandersnatch a chance to grab what landed on pypi | 21:15 |
| *** ldnunes has quit IRC | 21:16 | |
| fungi | true, we could make the proposal worker stop using our pypi mirror i suppose? | 21:16 |
| openstackgerrit | Timothy R. Chavez proposed openstack-infra/puppet-httpd: Revise offensive comment https://review.openstack.org/207222 | 21:16 |
| fungi | is that a desirable alteration to the current situation? | 21:16 |
| *** ekarlso has quit IRC | 21:16 | |
| clarkb | or just set PIP_foo in that job | 21:16 |
| fungi | ahh, that could also work i guess | 21:17 |
| clarkb | PIP_INDEX_URL iirc | 21:17 |
| fungi | though it does download a _lot_ of packages | 21:17 |
| fungi | if memory serves, it's basically retrieving everything in global-reqs and any dependencies thereof | 21:17 |
| * fungi double-checks | 21:18 | |
| fungi | yep, installs _everything_ | 21:19 |
| clarkb | I think that is correct | 21:19 |
| openstackgerrit | Mohammad Banikazemi proposed openstack-infra/irc-meetings: Add Kuryr weekly IRC meeting https://review.openstack.org/206324 | 21:19 |
| *** maishsk has joined #openstack-infra | 21:19 | |
| fungi | so, that's the trade-off. we can add a sleep, we can add a more complicated poll, or we can have the job download packages for the entirety of all our repos dependency chains from not-our-mirrors | 21:20 |
| fungi | i don't really have a preference as long as we can gain some consensus on which is least undesirable | 21:20 |
| fungi | or we can take dhellmann's route of manually generating and uploading a constraints update based on the thing being released (which i think gets hairy very quickly if there are multiple things being released in rapid succession) | 21:21 |
| *** alivigni has quit IRC | 21:23 | |
| *** e0ne has quit IRC | 21:24 | |
| jeblair | fungi: how does it get hairy? | 21:25 |
| jeblair | fungi: (despite its complexity, i think it's worth considering because it may be the most deterministic) | 21:25 |
| *** banix has quit IRC | 21:25 | |
| *** zz_dimtruck is now known as dimtruck | 21:27 | |
| EmilienM | jeblair: I've noticed an issue since our work on puppet/upgrade job : http://logs.openstack.org/58/206758/2/check/gate-puppet-keystone-puppet-beaker-rspec-upgrade-dsvm-trusty/0cfee85/console.html#_2015-07-29_19_48_34_635 | 21:28 |
| *** cbader has quit IRC | 21:28 | |
| EmilienM | it executes /usr/zuul-env/bin/zuul-cloner --cache-dir /opt/git --workspace /home/jenkins/workspace/gate-puppet-keystone-puppet-beaker-rspec-upgrade-dsvm-trusty/old --branch stable/kilo | 21:28 |
| jeblair | EmilienM: hrm, it doesn't seem to have any projects listed | 21:29 |
| fungi | jeblair: oh, i guess the manually-generate-and-propose-an-update method only gets hairy if you're trying to do that before releasing something | 21:30 |
| jeblair | EmilienM: i see the problem | 21:30 |
| EmilienM | not me | 21:30 |
| openstackgerrit | James E. Blair proposed openstack-infra/project-config: Fix typo in beaker-rspec-upgrade https://review.openstack.org/207227 | 21:30 |
| jeblair | EmilienM: ^ | 21:30 |
| EmilienM | ah indeed | 21:31 |
| EmilienM | jeblair: thank you | 21:31 |
| *** abregman is now known as abregman|ZzZz | 21:31 | |
| jeblair | EmilienM: np, and sorry :) | 21:31 |
| EmilienM | jeblair: I was reading zuul-cloner readme | 21:31 |
| *** johnbelamaric has joined #openstack-infra | 21:31 | |
| clarkb | we coukd just wait a day | 21:31 |
| fungi | jeblair: so it's definitely an alternative. we could drop the release pipeline run and just keep the post pipeline for the requirements repo | 21:31 |
| jeblair | clarkb: i keep asking myself why not that very question | 21:31 |
| clarkb | and if in a bind do it manually | 21:32 |
| *** cburroughs has quit IRC | 21:32 | |
| jeblair | what kind of bind would that be? | 21:32 |
| clarkb | jeblair: uncaught gate wedge due to not testing the entire world on constraints update | 21:32 |
| clarkb | I expect them to be fe wand far between | 21:32 |
| fungi | so far we've had people complain that the things they've released aren't being used in devstack jobs because the daily constraints proposal hasn't happened yet | 21:32 |
| clarkb | which is why I am happy to manually do it if necesary | 21:33 |
| jeblair | this is a system designed for _loosely_ coupled projects | 21:33 |
| fungi | so i suspect that's the main driver for wanting it | 21:33 |
| jeblair | we have a system for _tightly_ coupled projects, and it's not this one :) | 21:33 |
| *** achanda has joined #openstack-infra | 21:34 | |
| *** jsavak has quit IRC | 21:34 | |
| jeblair | clarkb: yeah, a specific constraint update can always be typed in by a human using hands on a keyboard :) | 21:34 |
| fungi | yep, so anyway, i'm not opposed to abandoning 202210. i have no real love for it, i simply added it on request | 21:35 |
| jeblair | or any input device, really | 21:35 |
| *** jsavak has joined #openstack-infra | 21:35 | |
| *** notnownikki has quit IRC | 21:36 | |
| jeblair | fungi: though we probably still want the post jobs on os/reqs itself? | 21:36 |
| fungi | and i think the requests for that have so far been due to some gate wedge being solved by a new release of something and the person getting that released not realizing that they needed the constraints update and then the delay in resolving teh wedge being blamed in part on the turn-around time for that constraints update to get proposed and merge | 21:36 |
| *** johnbelamaric has quit IRC | 21:36 | |
| fungi | to which the answer could simply be better education and adding more requirements-core reviewers to approve those | 21:37 |
| *** devkulkarni has quit IRC | 21:37 | |
| jeblair | yep, at least until we have zuulv3 running our actual mirror updates | 21:38 |
| clarkb | and in theory most of the gate wedges go away now | 21:38 |
| skylerberg | Does anyone know what the base url is for the gerrit REST API mentioned at https://review.openstack.org/Documentation/rest-api.html? | 21:38 |
| clarkb | skylerberg: https://review.openstack.org/ | 21:38 |
| *** achanda has quit IRC | 21:39 | |
| *** gordc has quit IRC | 21:39 | |
| *** sergmelikyan has quit IRC | 21:39 | |
| jeblair | (speaking of zuulv3, don't forget to vote https://review.openstack.org/164371) | 21:40 |
| *** jasondotstar has quit IRC | 21:41 | |
| skylerberg | clarkb: When I try https://review.openstack.org/projects for example, I get a 404. | 21:42 |
| fungi | skylerberg: trailing /? | 21:42 |
| *** tiswanso has joined #openstack-infra | 21:43 | |
| fungi | https://review.openstack.org/projects/ gets me a json blob, https://review.openstack.org/projects gives me a 404 | 21:43 |
| skylerberg | fungi: Yep | 21:43 |
| skylerberg | fungi: that was it | 21:43 |
| skylerberg | thanks clarkb, fungi! | 21:43 |
| clarkb | skylerberg: curl -X GET https://review.openstack.org/projects/ works fine from here | 21:43 |
| *** ekarlso has joined #openstack-infra | 21:43 | |
| fungi | anyway, i need to go get dinner and move some more of my junk. bbl | 21:43 |
| *** Guest29701 has quit IRC | 21:44 | |
| *** pc_m has quit IRC | 21:45 | |
| *** weshay has quit IRC | 21:46 | |
| clarkb | if we can get reviews for https://review.openstack.org/#/c/207187/1 and its child and the stack that begns at https://review.openstack.org/#/c/206175/2 we can begin making progress on the multinode front again (now that we are baiscally back to where we were before) | 21:47 |
| *** bhuvan has quit IRC | 21:48 | |
| *** tjones has joined #openstack-infra | 21:48 | |
| *** BharatK has quit IRC | 21:48 | |
| *** devkulkarni has joined #openstack-infra | 21:49 | |
| *** otter768 has joined #openstack-infra | 21:51 | |
| *** lhcheng_ has joined #openstack-infra | 21:52 | |
| *** tvanderwiel has quit IRC | 21:52 | |
| *** bswartz has joined #openstack-infra | 21:53 | |
| *** otter768 has quit IRC | 21:55 | |
| *** vahidh has joined #openstack-infra | 21:55 | |
| *** ihrachyshka has quit IRC | 21:56 | |
| *** thorst has quit IRC | 21:58 | |
| *** MarkAtwood has quit IRC | 21:59 | |
| *** sergmelikyan has joined #openstack-infra | 21:59 | |
| lifeless | fungi: ok so in the absence of that, care to tickle the periodic job? | 21:59 |
| *** prad has quit IRC | 21:59 | |
| lifeless | and uhm mordred or clarkb or someone - https://review.openstack.org/#/c/202210/ please | 22:00 |
| puranamr | ping | 22:01 |
| clarkb | lifeless: we were talking about not doing 202210 | 22:01 |
| clarkb | puranamr: pong | 22:01 |
| lifeless | clarkb: oh?! | 22:01 |
| *** markvoelker has quit IRC | 22:02 | |
| clarkb | lifeless: and rely on daily updates | 22:02 |
| *** Kennan2 has joined #openstack-infra | 22:02 | |
| kevinbenton | hi, was there an issue with the gate picking up changes ~12 hours ago? | 22:03 |
| *** erlon has quit IRC | 22:03 | |
| kevinbenton | i don't see any results for https://review.openstack.org/#/c/191048/ and it doesn't seem to be on http://status.openstack.org/zuul/ | 22:03 |
| *** Kennan has quit IRC | 22:03 | |
| lifeless | clarkb: why? | 22:03 |
| lifeless | clarkb: like - there's explicit interest in having releases get consumed rapidly | 22:03 |
| *** glauco_ has quit IRC | 22:03 | |
| clarkb | lifeless: because its complicated racy and unnecessary | 22:04 |
| lifeless | clarkb: and my goal is to eventually automate the entire thing so that its not justproposed, but proposed, tested, landed w/out human input | 22:04 |
| puranamr | clarkb: thanks for getting back, i am having a big trouble getting onto review.openstack.org inspite following all the steps | 22:04 |
| clarkb | we explicitly made aa system to not quikyl consume deps | 22:04 |
| clarkb | and now we are trying to work around it | 22:04 |
| puranamr | continuously getting permission denied | 22:04 |
| *** BharatK has joined #openstack-infra | 22:04 | |
| *** mriedem has quit IRC | 22:04 | |
| clarkb | puranamr: permission denied performing what action? | 22:04 |
| lifeless | clarkb: no, we made a system to only consume deps after verification, that is totally different to speed. | 22:04 |
| clarkb | lifeless: yes and we verify it on a timer | 22:05 |
| lifeless | clarkb: I don't understand | 22:05 |
| timrc | jeblair: Hi, we had a user attempt to use zuul-cloner in a post pipeline context and that failed due to missing zuul parameters. I take it zuul-cloner was not designed with cloning the post-merged repo in mind for additional processing, if that's the case, is that something zuul-cloner should be able to support? | 22:05 |
| clarkb | once a day we verify and propose an update | 22:05 |
| lifeless | clarkb: the periodic job is there as a *fallback* | 22:05 |
| clarkb | in the mean time nothing should break due to the constraints | 22:05 |
| lifeless | clarkb: it was not intended as the only or primary mechanism | 22:05 |
| clarkb | if something does break you can manually propose the fix | 22:05 |
| *** amitgandhinz has quit IRC | 22:05 | |
| clarkb | otherwise everything happens periodically | 22:05 |
| *** apmelton has quit IRC | 22:06 | |
| lifeless | clarkb: so, I have to go fly, but this is making no sense to me at all | 22:06 |
| * timrc thinks zuul-cloner should be able to take a revision in place of a change and patchset. | 22:06 | |
| clarkb | lifeless: read scrollback then? | 22:06 |
| clarkb | it all ahppened within the last hour | 22:06 |
| puranamr | git review -s or ssh or anything for that matter | 22:06 |
| lifeless | clarkb: at the summit latency was an explicit and significant concern from rel-mgmt | 22:06 |
| puranamr | clarkb: git review -s or ssh or anything for that matter | 22:06 |
| clarkb | lifeless: I dont understand that | 22:06 |
| mgagne | hmmm looks like Jenkins Git plugin doesn't like when slaves are taken offline by nodepool... a new build will be triggered so a workspace is created for polling and now I have infinite builds :-/ | 22:07 |
| lifeless | clarkb: fungi: if the worker is reused there will be a cache to make it faster | 22:07 |
| *** apmelton has joined #openstack-infra | 22:08 | |
| clarkb | puranamr: have you checked the username and ssh key? | 22:08 |
| *** jsavak has quit IRC | 22:08 | |
| lifeless | ok, so its still not making sense to me. The absolutely explicit request, nearly a conditional for doing this at all, at the summit, was getting new releases through the process in a timely manner | 22:08 |
| puranamr | clarkb: yes, followed all the instructions too | 22:08 |
| clarkb | lifeless: is 24hours not timely? | 22:09 |
| lifeless | no | 22:09 |
| clarkb | its good for translations and other updates | 22:09 |
| lifeless | this is not like translations | 22:09 |
| clarkb | ok we can increase the iteration frequency | 22:09 |
| clarkb | (it doesn't have to be daily could be hourly or every 5 minutes) | 22:09 |
| lifeless | we do requirements proposals on-changes | 22:09 |
| clarkb | lifeless: yes and they are much mor etrivial to deal with | 22:10 |
| puranamr | clarkb: there is small confusion, my user profile username says as ramap, so login as puranamr, so not sure | 22:10 |
| clarkb | as they do not deal with 1) large amounts of data 2) asynchronously updating mirrors | 22:10 |
| lifeless | clarkb: well, they do deal with the mirrors, but the latency in verifying anything is so much greater that folk don't notice | 22:10 |
| clarkb | lifeless: the current propoesd fix is a sleep which isn't correct | 22:10 |
| clarkb | so we could add that job but it would just be hand wavy | 22:10 |
| *** achanda has joined #openstack-infra | 22:11 | |
| puranamr | clarkb: apologies for the issue, i think i got a resolution | 22:11 |
| openstackgerrit | Michael Prokop proposed openstack-infra/jenkins-job-builder: Add ClamAV Plugin support https://review.openstack.org/207239 | 22:11 |
| lifeless | clarkb: sure, I get that. But the failure mode is that we don't get a proposal (vs e.g. getting a bad proposal) | 22:11 |
| clarkb | puranamr: ok its working now? | 22:11 |
| clarkb | lifeless: right which is no better than doing it daily | 22:11 |
| clarkb | so just do it daily... | 22:11 |
| lifeless | clarkb: so, I think doing it 3 hourly might be an ok compromise | 22:11 |
| lifeless | clarkb: we can't do it less than the time it takes to +A and go through validation and gate | 22:11 |
| puranamr | clarkb: used the id in the user-profile (ramap) and not puranamr and it worked! | 22:11 |
| lifeless | clarkb: otherwise the proposal will be recycled, and then the gate will abort it | 22:12 |
| *** devkulkarni has quit IRC | 22:12 | |
| clarkb | lifeless: which should be fine if there are updates | 22:12 |
| clarkb | and if there are not updates it would stand as is | 22:12 |
| lifeless | clarkb: except it has the same code, and thus the same bug, as the req updates we were looking at | 22:12 |
| lifeless | clarkb: and so its going to push up a new revision regardless | 22:12 |
| clarkb | lifeless: thta should and can be fixed | 22:13 |
| lifeless | until we address that | 22:13 |
| lifeless | sure | 22:13 |
| clarkb | andis independent | 22:13 |
| clarkb | completely unrelated | 22:13 |
| *** tiswanso has quit IRC | 22:13 | |
| *** d34dh0r53 has quit IRC | 22:13 | |
| *** odyssey4me has quit IRC | 22:13 | |
| lifeless | I dispute that; doing a 1 hour cycle on constraints updates would make them unlandable. Clearly a dependent variable. | 22:13 |
| *** d34dh0r53 has joined #openstack-infra | 22:13 | |
| clarkb | the diff thing is unrelated | 22:14 |
| lifeless | but I don't have time to rathole on that. Right now:- folk need to be told to come and ping here after doing some number of releases. | 22:14 |
| clarkb | lifeless: why? | 22:14 |
| lifeless | true or false? | 22:14 |
| *** hodos has quit IRC | 22:14 | |
| *** odyssey4me has joined #openstack-infra | 22:14 | |
| clarkb | can;t we just tell them to wait for tomorrow OR explicitly update constraints? | 22:14 |
| clarkb | so false | 22:14 |
| lifeless | because right now they bitch about not getting their release in use | 22:14 |
| *** yamamoto has joined #openstack-infra | 22:15 | |
| mgagne | found why... https://issues.jenkins-ci.org/browse/JENKINS-10131 | 22:15 |
| openstackgerrit | Clint Adams proposed stackforge/gertty: Support '-' as negation operator in query https://review.openstack.org/207240 | 22:15 |
| lifeless | put yourselve in their shoes for a bit. Think about their use cases. I'm sorry I'm not proxying it well enough, but I really Must Go. | 22:15 |
| lifeless | and - humans doing work we've already automated elsewhere is wasteful. | 22:15 |
| *** d34dh0r53 has quit IRC | 22:15 | |
| clarkb | the issue is automating this correctly in a way thta is more useful than the existing automation is not trivial | 22:16 |
| *** d34dh0r53 has joined #openstack-infra | 22:16 | |
| clarkb | so we can either A) spend too much time and effort fixing that or B) live with the existing automation whcih for the majority of cases works jst fine | 22:16 |
| clarkb | if there is a case where urgency is required we have a ay around it | 22:16 |
| *** Sukhdev_ has joined #openstack-infra | 22:19 | |
| *** yamamoto has quit IRC | 22:20 | |
| lifeless | or C) do something pragmatic that will do the right thing 95% of the time, and that we did in the previous analgous situation | 22:21 |
| clarkb | thats what we are doing... | 22:21 |
| lifeless | no; what was done previously was a trigger job w/ a sleep, AIUI. | 22:21 |
| lifeless | for the pip-mirror stuff way back | 22:21 |
| clarkb | nope | 22:22 |
| lifeless | right thing here is defined as 'get a proposal up in a timely manner for enabling the released packages' | 22:22 |
| clarkb | no sleeping or synchronizing was required then | 22:22 |
| lifeless | then I misunderstood something | 22:22 |
| lifeless | anyhow, you know the infra infrastructure way better than I | 22:22 |
| *** apmelton has quit IRC | 22:22 | |
| lifeless | But I have the user side requirements of 'don't make my life harder around getting releases to users' | 22:22 |
| *** d34dh0r53 has quit IRC | 22:23 | |
| *** mjturek1 has quit IRC | 22:23 | |
| lifeless | and I'm worried that we'll be reneging on a summit commitment made as a condition for getting this in | 22:23 |
| *** ashleighfarnham has quit IRC | 22:23 | |
| *** d34dh0r53 has joined #openstack-infra | 22:23 | |
| clarkb | I was not there to make any commitments, I can only offer my opinion on the matter as presented now | 22:24 |
| lifeless | which makes it important to communicate clearly and discuss with those same people if we're going to do something different | 22:24 |
| lifeless | sure | 22:24 |
| lifeless | I realise that | 22:24 |
| clarkb | and my opinion on 202210 is that is is not necessary and even worse broken | 22:24 |
| *** odyssey4me has quit IRC | 22:24 | |
| lifeless | which is why I've been trying to communicate how 'once a day' is a problem | 22:24 |
| lifeless | and 'just wait' is a problem | 22:24 |
| *** devkulkarni has joined #openstack-infra | 22:24 | |
| clarkb | I think if those are isseues we should not be using constraints... | 22:24 |
| *** apmelton has joined #openstack-infra | 22:25 | |
| *** odyssey4me has joined #openstack-infra | 22:25 | |
| *** bknudson has quit IRC | 22:25 | |
| *** banix has joined #openstack-infra | 22:25 | |
| clarkb | either you want careful control (constraints) or latest and greatest (no constrinats) | 22:25 |
| clarkb | merging both is problematic | 22:25 |
| clarkb | remember that even after the constraints proposal update it still has to be reviewed, approved, tested, and merged | 22:26 |
| clarkb | and thats going t obe much slower than a X hur periodic job to submit the changes | 22:27 |
| *** d34dh0r53 has quit IRC | 22:27 | |
| *** d34dh0r53 has joined #openstack-infra | 22:27 | |
| lifeless | thats why I said 3hr before | 22:28 |
| *** jasondotstar has joined #openstack-infra | 22:28 | |
| lifeless | and we ratholed on whether the diff logic needs fixing before that would work | 22:28 |
| lifeless | we have an expedited policy on reviews for these proposals - one reviewer, +2A as soon as it checks green. | 22:28 |
| *** armax has joined #openstack-infra | 22:29 | |
| clarkb | for right now the quickest turn around time would be ~11 hours | 22:29 |
| lifeless | careful control is entirely compatible with high frequency low latency system btw - in fact they work better than uncontrolled high frequency ones, because you can depend on stability | 22:29 |
| clarkb | 7 hours check, 4 hours gate | 22:29 |
| lifeless | because of the backlog? | 22:29 |
| clarkb | yes thats the current state | 22:29 |
| lifeless | ack | 22:29 |
| lifeless | so | 22:29 |
| jeblair | i don't see anything about instant updates in the task list which was cribbed from the whiteboard at vancouver | 22:30 |
| lifeless | jeblair: I am going off of dougs quite concerned discussion | 22:30 |
| openstackgerrit | Daniel Gonzalez Nothnagel proposed openstack-infra/devstack-gate: Set default for DEVSTACK_GATE_MULTIREGION to 1 https://review.openstack.org/205705 | 22:30 |
| openstackgerrit | Daniel Gonzalez Nothnagel proposed openstack-infra/devstack-gate: Multi-region dg support https://review.openstack.org/200309 | 22:30 |
| jeblair | i also don't remember that, so i must have been out of the room (or rather, in a room since it was outside) or something | 22:30 |
| lifeless | and to be clear, I'm not trying to jump up and down and cry event-based-event-based | 22:30 |
| jeblair | lifeless: ack, mental picture adjusted ;) | 22:30 |
| *** sigmavirus24 has quit IRC | 22:31 | |
| lifeless | the specific thing doug was concerned about was getting oslo lib releases to the rest of openstack promptly | 22:31 |
| lifeless | he suggested special casing them and not pinning them at all | 22:31 |
| *** Sherif_ has joined #openstack-infra | 22:31 | |
| lifeless | that concerned me since we are our own worst enemy there | 22:31 |
| lifeless | anyhow | 22:31 |
| *** d34dh0r53 has quit IRC | 22:32 | |
| jeblair | yeah, i guess i land on the side of "what, next day isn't promptly?" when it comes to software that consumes other software via releases | 22:32 |
| openstackgerrit | Alan Pevec proposed openstack-infra/release-tools: Ironic participates in stable point releases starting with Kilo https://review.openstack.org/203774 | 22:32 |
| lifeless | -> finish packing and fly. I would like to get this dialed up to 12 hourly once the backlog is recovered from. Doing anything right now would be poor form. | 22:32 |
| *** Sherif has quit IRC | 22:32 | |
| *** sigmavirus24 has joined #openstack-infra | 22:32 | |
| *** d34dh0r53 has joined #openstack-infra | 22:33 | |
| *** ashleighfarnham has joined #openstack-infra | 22:35 | |
| *** armax has quit IRC | 22:36 | |
| *** Sherif has joined #openstack-infra | 22:36 | |
| *** Sherif_ has quit IRC | 22:37 | |
| *** rossella_s has quit IRC | 22:37 | |
| *** rossella_s has joined #openstack-infra | 22:38 | |
| *** mwagner_lap has quit IRC | 22:38 | |
| *** abitha has joined #openstack-infra | 22:38 | |
| *** Sherif_ has joined #openstack-infra | 22:40 | |
| *** Sherif has quit IRC | 22:40 | |
| *** sdake has quit IRC | 22:40 | |
| *** sdake has joined #openstack-infra | 22:40 | |
| *** Sherif_ has quit IRC | 22:42 | |
| *** sdake has quit IRC | 22:42 | |
| *** Sherif has joined #openstack-infra | 22:42 | |
| *** sigmavirus24 is now known as sigmavirus24_awa | 22:43 | |
| *** bhuvan has joined #openstack-infra | 22:43 | |
| *** jaypipes has quit IRC | 22:43 | |
| *** sergmelikyan has quit IRC | 22:45 | |
| *** thorst has joined #openstack-infra | 22:45 | |
| *** devkulkarni has quit IRC | 22:45 | |
| *** esker has quit IRC | 22:46 | |
| *** bhuvan has quit IRC | 22:48 | |
| *** sergmelikyan has joined #openstack-infra | 22:48 | |
| *** samueldmq has quit IRC | 22:49 | |
| *** ericksonsantos has quit IRC | 22:50 | |
| *** ashleighfarnham has quit IRC | 22:50 | |
| *** Sherif has quit IRC | 22:51 | |
| *** Sherif has joined #openstack-infra | 22:51 | |
| *** austin81 has left #openstack-infra | 22:52 | |
| openstackgerrit | Merged openstack-infra/jenkins-job-builder: Support disable-publisher and both-html-text content-type in email-ext https://review.openstack.org/191534 | 22:56 |
| *** markvoelker_ has joined #openstack-infra | 22:57 | |
| nibalizer | ssssss | 23:00 |
| nibalizer | erp | 23:00 |
| openstackgerrit | Merged openstack-infra/jenkins-job-builder: Adding support for IvyTrigger Plugin https://review.openstack.org/200491 | 23:00 |
| *** rlandy has quit IRC | 23:00 | |
| *** rlandy has joined #openstack-infra | 23:00 | |
| openstackgerrit | Brandon Leonard proposed openstack-infra/jenkins-job-builder: Adds view functionality https://review.openstack.org/206178 | 23:05 |
| *** alexpilotti has quit IRC | 23:05 | |
| *** ZZelle has quit IRC | 23:07 | |
| ianw | fungi: re the 5 minute thing, i don't mind mich | 23:08 |
| ianw | much | 23:08 |
| *** ashleighfarnham has joined #openstack-infra | 23:08 | |
| *** tonytan4ever has quit IRC | 23:12 | |
| *** sergmelikyan has quit IRC | 23:13 | |
| *** maishsk has quit IRC | 23:17 | |
| openstackgerrit | Merged openstack-infra/puppet-vcsrepo: Boilerplate beaker-rspec files https://review.openstack.org/206425 | 23:17 |
| jeblair | whatever the gate pipeline is doing now, it has exceeded my ability to depict it graphically with some entry-level javascript | 23:17 |
| *** dimtruck is now known as zz_dimtruck | 23:18 | |
| *** puranamr has quit IRC | 23:18 | |
| *** thorst has quit IRC | 23:19 | |
| openstackgerrit | Merged openstack-infra/zuul: Fix IndexError: No item found with id u'origin/HEAD in tests https://review.openstack.org/206234 | 23:20 |
| openstackgerrit | Merged openstack-infra/zuul: Set build state after timing https://review.openstack.org/203836 | 23:20 |
| openstackgerrit | Merged openstack-infra/zuul: Don't use outdated repo object in reset https://review.openstack.org/203850 | 23:20 |
| *** jaosorior has quit IRC | 23:21 | |
| ianw | jeblair: it's like Christmas! just needs a few santas | 23:22 |
| *** devkulkarni has joined #openstack-infra | 23:22 | |
| jhesketh | Morning | 23:22 |
| jeblair | fungi, clarkb: i neglected to run 'nodepool dib-image-delete' earlier, so nodepool has helpfully re-uploaded images i deleted earlier. i'm doing a more thorough job now | 23:23 |
| openstackgerrit | Merged openstack-infra/project-config: Fix typo in beaker-rspec-upgrade https://review.openstack.org/207227 | 23:23 |
| clarkb | jeblair neat | 23:23 |
| openstackgerrit | Christopher Aedo proposed openstack-infra/project-config: Add repo for apps-catalog-ui https://review.openstack.org/207253 | 23:23 |
| *** ivar-lazzaro has quit IRC | 23:23 | |
| *** ashleighfarnham has quit IRC | 23:23 | |
| *** maishsk has joined #openstack-infra | 23:24 | |
| jeblair | jhesketh: https://review.openstack.org/148926 is the fix for the second-level cause of the zuul issues we saw earlier this week and last | 23:24 |
| *** maishsk has quit IRC | 23:24 | |
| jeblair | jhesketh: when that lands, i will restart it and then the next time it happens, we should see the actual error and traceback in logs | 23:24 |
| jeblair | jhesketh: also, if you can spare some brains to look at https://review.openstack.org/195184 that would be cool | 23:25 |
| jeblair | jhesketh: i just approved it a few mins ago, but even a post-merge approval would be good (we have time to revert before it hits images) | 23:25 |
| jeblair | er, post-merge review | 23:25 |
| *** kidchang has joined #openstack-infra | 23:26 | |
| *** kidchang has left #openstack-infra | 23:27 | |
| *** sergmelikyan has joined #openstack-infra | 23:27 | |
| openstackgerrit | Merged openstack-infra/puppet-pgsql_backup: Add .gitreview and module boilerplate https://review.openstack.org/206362 | 23:27 |
| *** annegentle has joined #openstack-infra | 23:28 | |
| *** dingyichen has joined #openstack-infra | 23:28 | |
| jeblair | okay, images extra-deleted now | 23:29 |
| openstackgerrit | Merged openstack-infra/puppet-accessbot: Boilerplate beaker-rspec files https://review.openstack.org/206363 | 23:30 |
| jhesketh | jeblair: sure, looking | 23:31 |
| *** oomichi has joined #openstack-infra | 23:31 | |
| openstackgerrit | Merged openstack-infra/puppet-vinz: Boilerplate beaker-rspec files https://review.openstack.org/206426 | 23:32 |
| *** jasondotstar has quit IRC | 23:33 | |
| *** hichihara has joined #openstack-infra | 23:36 | |
| *** Sherif is now known as sherif | 23:36 | |
| *** sherif has left #openstack-infra | 23:36 | |
| docaedo | jeblair: if you get a chance would like your feedback on 207253 (add repo for app catalog ui). I *think* I covered what needs covering :) | 23:37 |
| *** mwagner_lap has joined #openstack-infra | 23:37 | |
| docaedo | jeblair: but catching up on the scrollback, appears infra team has a whole lot going on at the moment, so I'll not pester | 23:37 |
| *** jamielennox is now known as jamielennox|away | 23:39 | |
| openstackgerrit | Merged openstack-infra/puppet-ansible: Boilerplate beaker-rspec files https://review.openstack.org/206364 | 23:39 |
| *** zz_dimtruck is now known as dimtruck | 23:40 | |
| openstackgerrit | Merged openstack-infra/puppet-apparmor: Boilerplate beaker-rspec files https://review.openstack.org/206365 | 23:41 |
| openstackgerrit | Merged openstack-infra/puppet-apps_site: Boilerplate beaker-rspec files https://review.openstack.org/206366 | 23:41 |
| *** ybathia has quit IRC | 23:43 | |
| *** bhuvan has joined #openstack-infra | 23:44 | |
| openstackgerrit | Merged openstack-infra/puppet-askbot: Boilerplate beaker-rspec files https://review.openstack.org/206367 | 23:45 |
| *** jasondotstar has joined #openstack-infra | 23:45 | |
| *** akshai has quit IRC | 23:46 | |
| *** armax has joined #openstack-infra | 23:47 | |
| *** darrenc is now known as darrenc_afk | 23:47 | |
| lifeless | right, airport loung. less stressed | 23:48 |
| lifeless | sorry for getting paniced before | 23:48 |
| *** bhuvan has quit IRC | 23:49 | |
| *** sergmelikyan has quit IRC | 23:50 | |
| *** lhcheng_ has left #openstack-infra | 23:50 | |
| openstackgerrit | Merged openstack-infra/puppet-asterisk: Boilerplate beaker-rspec files https://review.openstack.org/206368 | 23:51 |
| openstackgerrit | Merged openstack-infra/puppet-bandersnatch: Boilerplate beaker-rspec files https://review.openstack.org/206369 | 23:51 |
| openstackgerrit | Merged openstack-infra/puppet-bugdaystats: Boilerplate beaker-rspec files https://review.openstack.org/206370 | 23:51 |
| openstackgerrit | Merged openstack-infra/puppet-bup: Boilerplate beaker-rspec files https://review.openstack.org/206371 | 23:51 |
| openstackgerrit | Merged openstack-infra/zuul: cloner: make more zuul args optional https://review.openstack.org/195184 | 23:51 |
| openstackgerrit | Merged openstack-infra/zuul: show reconfiguration failures https://review.openstack.org/148926 | 23:51 |
| nibalizer | mmmmmm merges | 23:51 |
| *** Sherif has joined #openstack-infra | 23:51 | |
| *** otter768 has joined #openstack-infra | 23:51 | |
| *** Sherif is now known as sherif | 23:53 | |
| *** Kennan2 is now known as Kennan | 23:53 | |
| *** bharath_ has quit IRC | 23:53 | |
| *** amickus has quit IRC | 23:54 | |
| *** akshai has joined #openstack-infra | 23:55 | |
| *** otter768 has quit IRC | 23:56 | |
| *** ajmiller has quit IRC | 23:56 | |
| *** sergmelikyan has joined #openstack-infra | 23:57 | |
| *** sherif is now known as abdelwas | 23:59 | |
| * mordred hands nibalizer a merge orange | 23:59 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!