| david-lyle | I'm open to better ideas regarding the above patch ^^ | 00:00 |
|---|---|---|
| *** krotscheck has quit IRC | 00:01 | |
| david-lyle | pinning to an older version of firefox was the first attempt, but I did not find a common version between precise and trusty | 00:02 |
| *** e0ne has quit IRC | 00:02 | |
| david-lyle | so instead of introducing a fair amount of logic temporarily, I made the tests non-voting, at least that's the intent | 00:02 |
| *** jamesmcarthur has joined #openstack-infra | 00:03 | |
| * SpamapS adds some more super basic unit tests to shade and finds that os-client-config is not py3 clean.. doh | 00:04 | |
| *** che-arne has quit IRC | 00:05 | |
| *** oomichi has joined #openstack-infra | 00:05 | |
| *** MaxV has quit IRC | 00:05 | |
| openstackgerrit | Jan Hruban proposed openstack-infra/jenkins-job-builder: Support additional features in Build Flow project https://review.openstack.org/158084 | 00:06 |
| *** carl_baldwin has quit IRC | 00:06 | |
| openstackgerrit | Merged stackforge/gertty: Add support for starred changes https://review.openstack.org/155614 | 00:06 |
| openstackgerrit | Merged stackforge/gertty: Set priority of initial change sync to normal https://review.openstack.org/155615 | 00:07 |
| openstackgerrit | Merged stackforge/gertty: Fix reversing changes https://review.openstack.org/155773 | 00:07 |
| openstackgerrit | Maru Newby proposed openstack-infra/project-config: Add the neutron func job (nv) to devstack-gate https://review.openstack.org/159672 | 00:07 |
| openstackgerrit | Merged stackforge/gertty: Disable InsecureRequestWarning https://review.openstack.org/157259 | 00:07 |
| tchaypo | okay, so now that I’m back in the right part of the world | 00:08 |
| *** rfolco has joined #openstack-infra | 00:08 | |
| openstackgerrit | Merged stackforge/gertty: Security: Require config file to be mode 0600 https://review.openstack.org/157903 | 00:10 |
| jeblair | tchaypo: i left some suggestions on your keybinding changes | 00:11 |
| tchaypo | thanks :) | 00:11 |
| tchaypo | in other news | 00:11 |
| SpamapS | this makes me wonder.. do we graph code coverage anywhere? | 00:13 |
| tchaypo | we have two bastion hosts, one for each hp region | 00:14 |
| pleia2 | ok, back, time to cinder this up | 00:14 |
| tchaypo | how are we going to handle those being taken over by the infra team? | 00:14 |
| openstackgerrit | Clint 'SpamapS' Byrum proposed openstack-infra/shade: Add basic unit test for shade.openstack_cloud https://review.openstack.org/159674 | 00:15 |
| tchaypo | I’m thinking that the easiest thing to do would be for me to add ssh public keys to the shared account we’re currently using to log in | 00:16 |
| *** tiswanso has quit IRC | 00:16 | |
| tchaypo | and then the boxes can be emborgenated by puppet perhaps? | 00:16 |
| tchaypo | or at least, infra-root people can hammer the machines into a shape that they like | 00:16 |
| SpamapS | alright, my contribution to testing coverage done.. I must now retire to Hockey, Karate, and dinner. I bid you all farewell until 9pm. ;) | 00:16 |
| *** yamamoto has joined #openstack-infra | 00:17 | |
| *** david-lyle is now known as david-lyle_Afk | 00:22 | |
| *** david-lyle_Afk is now known as david-lyle_afk | 00:22 | |
| clarkb | pleia2 did you see that a single 1tb volume in lvm to start seemed reasonable? | 00:23 |
| clarkb | it gives us twice what we have now | 00:24 |
| *** markvoelker has quit IRC | 00:24 | |
| *** cody-somerville has quit IRC | 00:24 | |
| *** ghostpl_ has quit IRC | 00:25 | |
| pleia2 | clarkb: yeah, makes sense | 00:26 |
| * pleia2 creates cinder volume on ORD instead of DFW this time | 00:26 | |
| *** shashankhegde has joined #openstack-infra | 00:27 | |
| jeblair | pleia2: details :) | 00:27 |
| pleia2 | at least it was an easy mistake to recover from :) | 00:27 |
| *** cburgess has quit IRC | 00:28 | |
| *** cnesa has quit IRC | 00:28 | |
| *** cnesa has joined #openstack-infra | 00:28 | |
| *** cburgess has joined #openstack-infra | 00:28 | |
| *** mayurig has quit IRC | 00:29 | |
| *** gus has quit IRC | 00:29 | |
| *** mattoliverau has quit IRC | 00:29 | |
| *** mattoliverau has joined #openstack-infra | 00:29 | |
| *** EmilienM|afk has quit IRC | 00:29 | |
| *** markvoelker has joined #openstack-infra | 00:29 | |
| *** mwagner_lap has quit IRC | 00:29 | |
| *** EmilienM has joined #openstack-infra | 00:30 | |
| *** wenlock has quit IRC | 00:30 | |
| mordred | pleia2: you would have enjoyed when jeblair and I were standing up AFS | 00:31 |
| mordred | pleia2: I'm not going to say that that didn't happen 2 or 3 times that afternoon | 00:31 |
| *** gus has joined #openstack-infra | 00:31 | |
| *** tsg_ has quit IRC | 00:40 | |
| *** bhunter71 has quit IRC | 00:41 | |
| *** ZZelle_ has quit IRC | 00:45 | |
| openstackgerrit | Devananda van der Veen proposed openstack-infra/reviewstats: Overdue update for the ironic project group https://review.openstack.org/159680 | 00:48 |
| *** yamamoto has quit IRC | 00:49 | |
| jeblair | anteaya, reed: i replaced the gerritworkflow wiki page with a link to the infra manual (i think most links were updated to point to it a while ago, and i think enough time has passed to make this change) | 00:50 |
| jeblair | alse gerritjenkinsgit | 00:50 |
| openstackgerrit | Elizabeth K. Joseph proposed openstack-infra/system-config: Add instructions for vgcreate to sysadmin docs https://review.openstack.org/159681 | 00:50 |
| mordred | ooh. docs | 00:50 |
| jogo | clarkb: so wtf trove http://logs.openstack.org/47/159247/1/check/check-tempest-dsvm-full-juno/4f19a79/logs/devstacklog.txt.gz#_2015-02-26_20_46_04_484 | 00:54 |
| jogo | trove on stable/juno is pulling http://tarballs.openstack.org/trove/images/ubuntu_mysql.qcow2/ubuntu_mysql.qcow2 | 00:55 |
| *** yamamoto has joined #openstack-infra | 00:55 | |
| *** reed has quit IRC | 00:55 | |
| jogo | which takes an hour | 00:55 |
| mordred | yeah. I believe that's what we asked them to do | 00:55 |
| clarkb | it should be cached | 00:56 |
| jogo | doesn't appear to be on juno though | 00:56 |
| *** vilobhmm has quit IRC | 00:58 | |
| *** achanda has quit IRC | 00:58 | |
| *** SumitNaiksatam has left #openstack-infra | 00:59 | |
| *** vilobhmm has joined #openstack-infra | 00:59 | |
| *** SumitNaiksatam has joined #openstack-infra | 00:59 | |
| *** dangers is now known as dangers_away | 01:00 | |
| *** vilobhmm has quit IRC | 01:00 | |
| *** vilobhmm has joined #openstack-infra | 01:00 | |
| *** dannywilson has quit IRC | 01:00 | |
| jogo | so the location of the image was changed 4 months ago in https://review.openstack.org/#/c/131002/ on master | 01:01 |
| *** dims has joined #openstack-infra | 01:01 | |
| pleia2 | clarkb: we're putting backups in /home/bup-$server? D: | 01:03 |
| clarkb | pleia2: ya, but I think we can change that if it makes mounting filesystems easier | 01:03 |
| clarkb | pleia2: or maybe symlink only other filesystem? I should actually think about this for a bit | 01:03 |
| pleia2 | I don't think we want to put all of /home on a cinder volume | 01:03 |
| pleia2 | clarkb: in the meantime, if you want to hop on 23.253.166.59 and confirm my vg and lv looks ok, much appreciated | 01:04 |
| *** sigmavirus24_awa is now known as sigmavirus24 | 01:06 | |
| pleia2 | updating fstab now too | 01:07 |
| pleia2 | er, actually once I know where to mount it | 01:07 |
| clarkb | pgs and vgs output looks correct to me | 01:08 |
| *** Sukhdev_ has quit IRC | 01:08 | |
| pleia2 | great | 01:08 |
| *** bknudson has joined #openstack-infra | 01:10 | |
| clarkb | pleia2: so here's what I think we should probably do for mounting. Put the mount on /opt/backups or similar since we tend to use /opt for extra volumes. Then when we create the users create their homedires under that path | 01:10 |
| clarkb | jeblair: ^ that seem reasonable? | 01:10 |
| clarkb | pleia2: we would then only need to update the docs as creating the user is a manual step iirc | 01:10 |
| openstackgerrit | Swaminathan Vasudevan proposed openstack-infra/devstack-gate: Configure sub-nodes for Neutron DVR aiopcpu https://review.openstack.org/158525 | 01:11 |
| *** yamamoto has quit IRC | 01:11 | |
| *** armax has joined #openstack-infra | 01:11 | |
| openstackgerrit | Steve Baker proposed openstack-infra/project-config: Create and upload heat test images to tarballs https://review.openstack.org/159686 | 01:12 |
| *** bdpayne has quit IRC | 01:13 | |
| *** yamamoto has joined #openstack-infra | 01:14 | |
| *** bhunter71 has joined #openstack-infra | 01:18 | |
| *** erlon_away has quit IRC | 01:21 | |
| *** SumitNaiksatam has quit IRC | 01:25 | |
| openstackgerrit | greghaynes proposed openstack-infra/project-config: Support custom static nameserver during build https://review.openstack.org/157554 | 01:28 |
| *** SotK has quit IRC | 01:28 | |
| *** baoli_ has quit IRC | 01:28 | |
| *** SotK has joined #openstack-infra | 01:28 | |
| *** ddieterly has joined #openstack-infra | 01:29 | |
| *** shashankhegde has quit IRC | 01:29 | |
| greghaynes | clarkb: ^ | 01:30 |
| greghaynes | for now, ill break out the nameserver to its own element later | 01:30 |
| *** ddieterl_ has joined #openstack-infra | 01:30 | |
| *** markvoelker has quit IRC | 01:32 | |
| *** sdake_ has quit IRC | 01:32 | |
| *** ChuckC has quit IRC | 01:33 | |
| *** yamamoto has quit IRC | 01:33 | |
| *** marun has quit IRC | 01:33 | |
| *** ddieterly has quit IRC | 01:34 | |
| openstackgerrit | Michael Johnson proposed openstack-infra/git-review: Choose correct branch for rebase when submitting https://review.openstack.org/158877 | 01:40 |
| *** otter768 has joined #openstack-infra | 01:41 | |
| *** armax has quit IRC | 01:42 | |
| bhunter71 | If I was to, hypothetically, have lost my gerrit cinder volume, what did I lose? Is there state in there beyond ~gerrit2/review_site/git that I care about? (I do not think so, and I can recover all git repos) | 01:43 |
| mordred | bhunter71: no - other state for the gerrit is in the mysql db | 01:45 |
| mordred | the only interesting stuff is the git dirs and the db | 01:45 |
| bhunter71 | mordred: thanks, that's my impression. here goes nothing | 01:45 |
| mordred | woot | 01:45 |
| *** otter768 has quit IRC | 01:45 | |
| *** rm_work is now known as rm_work|away | 01:46 | |
| *** yamamoto has joined #openstack-infra | 01:50 | |
| *** tsg has joined #openstack-infra | 01:51 | |
| pleia2 | clarkb: let's can pick up backup organization in the morning, crisis of space on current system is triaged and I'm heading off to evening $event soon | 01:56 |
| clarkb | pleia2: yup have fun | 01:56 |
| pleia2 | clarkb: but one last thing, does hpcloud have cinder volumes? or are we just using a bigger flavor there? | 01:56 |
| *** armax has joined #openstack-infra | 01:56 | |
| clarkb | pleia2: it does have cinder volumes, I don't know what our quota is like though. | 01:56 |
| pleia2 | clarkb: ok, maybe we'll explore that in the AM too then :) | 01:57 |
| *** achanda has joined #openstack-infra | 01:58 | |
| *** SumitNaiksatam has joined #openstack-infra | 01:59 | |
| *** sdake_ has joined #openstack-infra | 02:03 | |
| jeblair | clarkb, pleia2: user homedirs in opt sounds good | 02:09 |
| *** changbl has joined #openstack-infra | 02:09 | |
| nibalizer | cinder volumes yes | 02:10 |
| nibalizer | capped at 2T size tho | 02:10 |
| clarkb | nibalizer: thats fine, we are capped at 1TB by rax, so we lvm a bunch of them together when we need more | 02:11 |
| *** tiswanso has joined #openstack-infra | 02:11 | |
| nibalizer | haha | 02:11 |
| nibalizer | do you raid on them too? | 02:11 |
| nibalizer | gozer raids on cinder because trust in hpcloud is low | 02:11 |
| clarkb | well we didn't because there was and is no way (that we can figure) to tell cinder that these volumes should come from different hosts | 02:11 |
| clarkb | so if you raid and backend goes awa all your volumes could go away too | 02:12 |
| clarkb | so why raid | 02:12 |
| *** rmcall has joined #openstack-infra | 02:13 | |
| *** Ryan_Lane has quit IRC | 02:13 | |
| clarkb | also if anyone knows how to deal with ^ I would love to be educated :) | 02:15 |
| *** jamesmcarthur has quit IRC | 02:16 | |
| *** sigmavirus24 is now known as sigmavirus24_awa | 02:17 | |
| *** vilobhmm has quit IRC | 02:17 | |
| *** vilobhmm has joined #openstack-infra | 02:17 | |
| *** vilobhmm has quit IRC | 02:18 | |
| *** vilobhmm has joined #openstack-infra | 02:18 | |
| openstackgerrit | Julia Kreger proposed openstack-infra/shade: Add behavior to enable ironic noauth mode https://review.openstack.org/159609 | 02:18 |
| openstackgerrit | Clark Boylan proposed openstack-infra/project-config: Use sudo when manipulating git for image list https://review.openstack.org/159693 | 02:21 |
| clarkb | jogo: mtreinish ^ that is the simple fix | 02:21 |
| clarkb | I am about to ru nthat locally and see how it does | 02:22 |
| clarkb | except now my test machine is hosed and its late | 02:25 |
| clarkb | I will pick that up in the morning | 02:25 |
| *** baoli has joined #openstack-infra | 02:25 | |
| *** rwsu is now known as rwsu-afk | 02:26 | |
| *** gyee has quit IRC | 02:31 | |
| openstackgerrit | Joshua Hesketh proposed openstack-infra/zuul: Update the description for grey icons on status pg https://review.openstack.org/154690 | 02:31 |
| *** ivar-laz_ has joined #openstack-infra | 02:32 | |
| *** jamesmcarthur has joined #openstack-infra | 02:33 | |
| *** ivar-lazzaro has quit IRC | 02:35 | |
| *** sputnik13 has quit IRC | 02:36 | |
| *** vilobhmm has quit IRC | 02:36 | |
| *** vilobhmm has joined #openstack-infra | 02:36 | |
| *** ivar-laz_ has quit IRC | 02:36 | |
| *** vilobhmm has quit IRC | 02:37 | |
| *** woodster_ has quit IRC | 02:40 | |
| openstackgerrit | Michael Johnson proposed openstack-infra/git-review: Choose correct branch for rebase when submitting https://review.openstack.org/158877 | 02:41 |
| *** carl_baldwin has joined #openstack-infra | 02:42 | |
| openstackgerrit | Merged openstack-infra/gitdm: Add Adam Sheldon to email mapp file. https://review.openstack.org/159274 | 02:47 |
| armax | anyone knows how to nudge https://review.openstack.org/#/c/159199/ in? | 02:48 |
| asselin | qick question (hopefully) does infra use the same nodepool images for juno and master? | 02:49 |
| *** dkehn has quit IRC | 02:52 | |
| *** yamamoto has quit IRC | 02:53 | |
| *** weshay has quit IRC | 02:56 | |
| *** carl_baldwin has quit IRC | 02:58 | |
| *** dkranz has quit IRC | 02:58 | |
| *** yamamoto has joined #openstack-infra | 02:59 | |
| openstackgerrit | Merged openstack-infra/project-config: Improve gate-check logging https://review.openstack.org/159578 | 03:00 |
| *** sdake_ has quit IRC | 03:02 | |
| *** dkehn has joined #openstack-infra | 03:03 | |
| *** yamahata has quit IRC | 03:05 | |
| *** achanda has quit IRC | 03:05 | |
| *** unicell has quit IRC | 03:06 | |
| *** mwagner_lap has joined #openstack-infra | 03:06 | |
| *** patrickeast has quit IRC | 03:07 | |
| *** achanda has joined #openstack-infra | 03:08 | |
| *** rmcall has quit IRC | 03:10 | |
| *** rfolco has quit IRC | 03:10 | |
| *** rmcall has joined #openstack-infra | 03:12 | |
| *** carl_baldwin has joined #openstack-infra | 03:18 | |
| *** stevemar has joined #openstack-infra | 03:18 | |
| mordred | asselin: yes | 03:22 |
| *** sdake_ has joined #openstack-infra | 03:23 | |
| *** SumitNaiksatam has quit IRC | 03:24 | |
| *** rmcall has quit IRC | 03:25 | |
| *** carl_baldwin has quit IRC | 03:25 | |
| *** jyuso1 has joined #openstack-infra | 03:25 | |
| *** dmsimard_away is now known as dmsimard | 03:31 | |
| *** jamesmcarthur has quit IRC | 03:37 | |
| gus | Does anyone have info on why neutron pylint checks were failing a few days ago and moved to non-voting? | 03:37 |
| gus | I'm equally confused how it was made non-voting - I _think_ layout.yaml still says it's voting (but I've interpreted that file incorrectly in the past) | 03:38 |
| *** sdake_ has quit IRC | 03:39 | |
| *** sabeen has joined #openstack-infra | 03:40 | |
| *** achanda has quit IRC | 03:40 | |
| *** jamesmcarthur has joined #openstack-infra | 03:40 | |
| *** koolhead17 has joined #openstack-infra | 03:40 | |
| *** dims has quit IRC | 03:41 | |
| *** otter768 has joined #openstack-infra | 03:42 | |
| gus | aha novaclient broke it somehow: I found the change where it was removed from gate (but left as a check, which threw me): I10d68b976d81be1575df50ba009c8407b54e10ff | 03:43 |
| *** crc32 has joined #openstack-infra | 03:46 | |
| *** otter768 has quit IRC | 03:46 | |
| *** armax has quit IRC | 03:50 | |
| *** hdd has joined #openstack-infra | 03:51 | |
| *** dannywilson has joined #openstack-infra | 03:57 | |
| *** ddieterl_ has quit IRC | 03:58 | |
| *** achanda has joined #openstack-infra | 03:59 | |
| *** SumitNaiksatam has joined #openstack-infra | 03:59 | |
| *** dannywilson has quit IRC | 04:02 | |
| openstackgerrit | Richard Jones proposed openstack/requirements: Add new prettytable requirement for nova https://review.openstack.org/159706 | 04:05 |
| *** achanda has quit IRC | 04:07 | |
| *** sdake_ has joined #openstack-infra | 04:10 | |
| *** baoli has quit IRC | 04:17 | |
| *** armax has joined #openstack-infra | 04:17 | |
| *** baoli has joined #openstack-infra | 04:17 | |
| *** tiswanso has quit IRC | 04:21 | |
| *** annegentle has joined #openstack-infra | 04:30 | |
| *** unicell has joined #openstack-infra | 04:32 | |
| *** dmsimard is now known as dmsimard_away | 04:34 | |
| *** BharatK has joined #openstack-infra | 04:34 | |
| *** jamesmcarthur has quit IRC | 04:37 | |
| *** dims has joined #openstack-infra | 04:42 | |
| *** yfried has joined #openstack-infra | 04:43 | |
| *** yfried|afk has quit IRC | 04:43 | |
| *** jerryz has joined #openstack-infra | 04:46 | |
| *** dims has quit IRC | 04:47 | |
| *** deepakcs has joined #openstack-infra | 04:48 | |
| *** mrda is now known as mrda-away | 04:50 | |
| *** annegentle has quit IRC | 04:51 | |
| *** achanda has joined #openstack-infra | 04:55 | |
| *** sdake_ has quit IRC | 04:56 | |
| *** ddieterly has joined #openstack-infra | 04:58 | |
| *** carl_baldwin has joined #openstack-infra | 04:59 | |
| *** achanda has quit IRC | 04:59 | |
| *** markvoelker has joined #openstack-infra | 05:01 | |
| *** unicell has quit IRC | 05:01 | |
| *** unicell has joined #openstack-infra | 05:03 | |
| *** ddieterly has quit IRC | 05:03 | |
| gus | This change to project-config seems stuck: https://review.openstack.org/#/c/159199/ | 05:04 |
| gus | It has workflow+1 and passed jenkins check (but not gate?) and just seems to be sitting there. | 05:04 |
| gus | Is this waiting on something that will resolve itself, or does it need a human to rescue it? | 05:05 |
| *** coolsvap_ is now known as coolsvap | 05:08 | |
| deepakcs | gus, i have seen that behaviour before for one other patch, it probably looks liek if the core reviewer gives workflow +1 and then code review +2, it doesn't consider the workflow +1 | 05:08 |
| deepakcs | gus, the right order is coder review +2 and then workflow +1, then it auto merges.. | 05:08 |
| deepakcs | gus, thats my observation tho' , not sure if gerrit is designed to do that explicitly or its a bug :) | 05:08 |
| gus | deepakcs: huh. So I need to get clarkb to +2 separately to his workflow+1? | 05:10 |
| gus | clarkb: ^ ? | 05:10 |
| *** achanda has joined #openstack-infra | 05:11 | |
| deepakcs | gus, as i said, i am not sure, but i have seen that behaviour before | 05:15 |
| deepakcs | gus, but ur patch also said "Need verified", canu try giving a +1 for verified urself ? | 05:15 |
| StevenK | Verified is only set by Jenkins | 05:16 |
| *** baoli has quit IRC | 05:19 | |
| gus | yeah, I think we're waiting for the jenkins gate check - I just didn't expect that to take 30+ hours, so I assume something has gone wrong... | 05:19 |
| StevenK | gus: But it also doesn't show up on status.o.o | 05:19 |
| *** carl_baldwin has quit IRC | 05:24 | |
| pleia2 | hm, looks like it didnt finish/complete in the gate queue for some reason, I can give it a try | 05:25 |
| pleia2 | it's in the queue now, we'll see how it goes | 05:26 |
| *** achanda has quit IRC | 05:28 | |
| *** markvoelker has quit IRC | 05:33 | |
| *** markvoelker has joined #openstack-infra | 05:33 | |
| openstackgerrit | Merged openstack-infra/project-config: Remove explicit pylint job for Neutron https://review.openstack.org/159199 | 05:34 |
| pleia2 | gus: tada ^^ | 05:34 |
| pleia2 | dinner now | 05:34 |
| gus | \o/ - thanks | 05:34 |
| StevenK | When it doubt, throw more Workflow+1 at it | 05:36 |
| jerryz | pleia2: would sequence matter? | 05:36 |
| jerryz | pleia2: we also met similar problem. the core reviewer first give it a workflow +1 and later a review +2 | 05:37 |
| gus | fwiw, this change also had a Depends-On. The dependent change was submitted after the original workflow+1, perhaps the gate check doesn't get (re)queued correctly when a depends-on dependent has been submitted? | 05:37 |
| *** markvoelker has quit IRC | 05:37 | |
| jerryz | pleia2: workaround it by taking out the workflow +1 and redoing it. | 05:38 |
| StevenK | gus: That sounds plausible | 05:39 |
| jerryz | gus: i didn't see a dependency in the patch you posted | 05:41 |
| gus | Yeah I didn't notice it either until just now, but https://review.openstack.org/#/c/159199/ has a Depends-On | 05:42 |
| *** otter768 has joined #openstack-infra | 05:42 | |
| *** sdake_ has joined #openstack-infra | 05:43 | |
| *** otter768 has quit IRC | 05:47 | |
| *** Krast has joined #openstack-infra | 05:52 | |
| *** sdake_ has quit IRC | 05:54 | |
| *** abhirc has quit IRC | 05:55 | |
| *** achanda has joined #openstack-infra | 05:59 | |
| *** ddieterly has joined #openstack-infra | 06:00 | |
| *** jerryz_ has joined #openstack-infra | 06:00 | |
| *** vilobhmm has joined #openstack-infra | 06:00 | |
| *** vilobhmm1 has joined #openstack-infra | 06:03 | |
| *** markvoelker has joined #openstack-infra | 06:04 | |
| *** ddieterly has quit IRC | 06:04 | |
| *** vilobhmm has quit IRC | 06:05 | |
| *** hdd has quit IRC | 06:05 | |
| *** armax has quit IRC | 06:06 | |
| *** armax has joined #openstack-infra | 06:08 | |
| *** frickler has quit IRC | 06:08 | |
| *** markvoelker has quit IRC | 06:09 | |
| *** tobiash has quit IRC | 06:09 | |
| *** ildikov has quit IRC | 06:12 | |
| *** carl_baldwin has joined #openstack-infra | 06:13 | |
| *** oomichi has quit IRC | 06:13 | |
| *** deepakcs has quit IRC | 06:14 | |
| *** mayurig has joined #openstack-infra | 06:17 | |
| *** mayurig has quit IRC | 06:17 | |
| *** jkraj has joined #openstack-infra | 06:18 | |
| *** tobiash has joined #openstack-infra | 06:19 | |
| *** carl_baldwin has quit IRC | 06:20 | |
| openstackgerrit | Trinath Somanchi proposed openstack-infra/project-config: Add project 'networking-freescale' to Stackforge. https://review.openstack.org/159362 | 06:22 |
| *** resker has joined #openstack-infra | 06:26 | |
| *** esker has quit IRC | 06:26 | |
| *** melwitt has quit IRC | 06:29 | |
| *** dims has joined #openstack-infra | 06:31 | |
| *** Mmike has quit IRC | 06:32 | |
| *** dangers_away has quit IRC | 06:32 | |
| *** dangers_away has joined #openstack-infra | 06:33 | |
| *** Mmike has joined #openstack-infra | 06:36 | |
| *** dims has quit IRC | 06:36 | |
| *** resker has quit IRC | 06:37 | |
| *** camunoz has quit IRC | 06:38 | |
| *** Mmike has quit IRC | 06:38 | |
| *** Mmike has joined #openstack-infra | 06:38 | |
| *** ghostpl_ has joined #openstack-infra | 06:43 | |
| *** cnesa1 has joined #openstack-infra | 06:44 | |
| *** woodster_ has joined #openstack-infra | 06:44 | |
| *** esker has joined #openstack-infra | 06:45 | |
| *** cnesa has quit IRC | 06:46 | |
| *** HeOS has quit IRC | 06:48 | |
| *** mrunge has joined #openstack-infra | 06:49 | |
| *** koolhead17 has quit IRC | 06:50 | |
| *** esker has quit IRC | 06:50 | |
| *** cnesa1 has quit IRC | 06:53 | |
| *** tsg_ has joined #openstack-infra | 06:56 | |
| *** sabeen has quit IRC | 06:57 | |
| *** tsg has quit IRC | 06:59 | |
| *** ddieterly has joined #openstack-infra | 07:00 | |
| *** e0ne has joined #openstack-infra | 07:01 | |
| *** BharatK has quit IRC | 07:02 | |
| *** markvoelker has joined #openstack-infra | 07:05 | |
| *** ddieterly has quit IRC | 07:05 | |
| *** scheuran has joined #openstack-infra | 07:08 | |
| *** markvoelker has quit IRC | 07:09 | |
| *** ghostpl_ has quit IRC | 07:12 | |
| *** SumitNaiksatam has quit IRC | 07:19 | |
| *** SumitNaiksatam has joined #openstack-infra | 07:21 | |
| *** cnesa has joined #openstack-infra | 07:25 | |
| *** abhishek_cb has joined #openstack-infra | 07:26 | |
| *** tnovacik has joined #openstack-infra | 07:33 | |
| *** crc32 has quit IRC | 07:37 | |
| *** achanda has quit IRC | 07:38 | |
| *** BharatK has joined #openstack-infra | 07:39 | |
| *** _nadya_ has joined #openstack-infra | 07:42 | |
| *** stevemar has quit IRC | 07:43 | |
| *** _nadya_ has quit IRC | 07:43 | |
| *** otter768 has joined #openstack-infra | 07:43 | |
| *** _nadya_ has joined #openstack-infra | 07:44 | |
| *** achanda has joined #openstack-infra | 07:45 | |
| *** deepakcs has joined #openstack-infra | 07:47 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/infra-manual: Add reference to IRC services document https://review.openstack.org/159740 | 07:47 |
| *** otter768 has quit IRC | 07:48 | |
| *** achanda has quit IRC | 07:50 | |
| *** armax has quit IRC | 07:52 | |
| *** dmellado has quit IRC | 07:52 | |
| *** vilobhmm1 has quit IRC | 07:54 | |
| *** dmellado has joined #openstack-infra | 07:54 | |
| *** ddieterly has joined #openstack-infra | 08:01 | |
| *** bhunter71 has quit IRC | 08:04 | |
| *** ddieterly has quit IRC | 08:06 | |
| *** dtantsur|afk is now known as dtantsur | 08:07 | |
| *** mpavone has joined #openstack-infra | 08:09 | |
| *** mrmartin has joined #openstack-infra | 08:10 | |
| *** ildikov has joined #openstack-infra | 08:10 | |
| *** jcoufal has joined #openstack-infra | 08:10 | |
| *** MaxV has joined #openstack-infra | 08:11 | |
| *** frickler has joined #openstack-infra | 08:11 | |
| *** MaxV has quit IRC | 08:12 | |
| *** tgohad has joined #openstack-infra | 08:13 | |
| *** tsg_ has quit IRC | 08:15 | |
| *** doude has joined #openstack-infra | 08:15 | |
| *** e0ne has quit IRC | 08:16 | |
| *** mase_home has quit IRC | 08:16 | |
| *** matrohon has joined #openstack-infra | 08:23 | |
| *** abhishek_cb has quit IRC | 08:26 | |
| *** yfried has quit IRC | 08:27 | |
| *** AJaeger has joined #openstack-infra | 08:27 | |
| *** ChuckC has joined #openstack-infra | 08:27 | |
| openstackgerrit | Michael Krotscheck proposed openstack-infra/storyboard: Exclude pyMySQL v0.6.4 https://review.openstack.org/159564 | 08:29 |
| openstackgerrit | Michael Krotscheck proposed openstack-infra/storyboard: Reduce the number of timezones under test. https://review.openstack.org/159576 | 08:29 |
| openstackgerrit | Michael Krotscheck proposed openstack-infra/storyboard: Added milestones https://review.openstack.org/152500 | 08:29 |
| *** salv-orlando has joined #openstack-infra | 08:34 | |
| openstackgerrit | Michael Krotscheck proposed openstack-infra/storyboard: User abort call instead of returning response https://review.openstack.org/152515 | 08:36 |
| *** ildikov has quit IRC | 08:37 | |
| *** dtantsur is now known as dtantsur|bbl | 08:43 | |
| *** chlong has quit IRC | 08:44 | |
| openstackgerrit | Michael Krotscheck proposed openstack-infra/storyboard: Replaced python-crontab with apscheduler https://review.openstack.org/158955 | 08:46 |
| *** nfedotov has joined #openstack-infra | 08:47 | |
| *** MaxV has joined #openstack-infra | 08:52 | |
| *** liam_ has joined #openstack-infra | 08:54 | |
| *** dizquierdo has joined #openstack-infra | 08:55 | |
| *** liam_ has quit IRC | 08:55 | |
| *** mpaolino has joined #openstack-infra | 08:56 | |
| *** jistr has joined #openstack-infra | 08:56 | |
| *** yamamoto has quit IRC | 08:58 | |
| deepakcs | AJaeger, ping | 08:58 |
| deepakcs | anteaya, ping | 08:59 |
| AJaeger | morning, deepakcs | 08:59 |
| deepakcs | AJaeger, morning | 08:59 |
| deepakcs | AJaeger, BharatK started 2 experimental jobs in https://review.openstack.org/#/c/159711/1 | 09:00 |
| *** gmann has quit IRC | 09:00 | |
| deepakcs | AJaeger, but we only see 1 job in the jenkins job home page @ https://jenkins07.openstack.org/job/gate-tempest-dsvm-full-glusterfs/ | 09:00 |
| deepakcs | AJaeger, why the discrepancy ? | 09:00 |
| *** mpaolino has quit IRC | 09:00 | |
| *** mpaolino has joined #openstack-infra | 09:01 | |
| deepakcs | AJaeger, from the patch if u follow the job links, they points us to 2 different jobs, but why doesn't jenkins page show 2 jobs , since it run 2 times | 09:01 |
| AJaeger | click on the "toggle ci" to see all results | 09:02 |
| *** ddieterly has joined #openstack-infra | 09:02 | |
| *** markus_z has joined #openstack-infra | 09:02 | |
| *** HeOS has joined #openstack-infra | 09:03 | |
| *** yamamoto has joined #openstack-infra | 09:03 | |
| *** ddieterly has quit IRC | 09:06 | |
| deepakcs | AJaeger, didn't get u.. i did use toggle CI | 09:07 |
| deepakcs | AJaeger, 'guess you didn't get my Q.. The patch ran glusterfs CI job twice, but jenkins job page shows just 1, why ? | 09:07 |
| *** markvoelker has joined #openstack-infra | 09:07 | |
| *** woodster_ has quit IRC | 09:10 | |
| *** andreykurilin_ has joined #openstack-infra | 09:12 | |
| *** zz_johnthetubagu is now known as johnthetubaguy | 09:12 | |
| *** koolhead17 has joined #openstack-infra | 09:12 | |
| *** markvoelker has quit IRC | 09:13 | |
| *** cnesa1 has joined #openstack-infra | 09:14 | |
| AJaeger | deepakcs: I see it twice at 06:31 and 7:56 | 09:14 |
| AJaeger | but only after toggle CI | 09:14 |
| deepakcs | AJaeger, psl visit https://jenkins07.openstack.org/job/gate-tempest-dsvm-full-glusterfs/ | 09:14 |
| deepakcs | AJaeger, this show just 1 job, so was curious why it shows 1 | 09:15 |
| *** cnesa has quit IRC | 09:15 | |
| AJaeger | deepakcs: use the link from https://review.openstack.org/#/c/159711/1 | 09:15 |
| deepakcs | AJaeger, i see the 2 run in the patch, but my Q is on the jenkins page as above (Not on the patch page) | 09:15 |
| AJaeger | jobs are executed on several jenkins systems | 09:15 |
| AJaeger | results are http://logs.openstack.org/11/159711/1/experimental/gate-tempest-dsvm-full-glusterfs/b2cb37e/ and http://logs.openstack.org/11/159711/1/experimental/gate-tempest-dsvm-full-glusterfs/f8f267f/ | 09:16 |
| deepakcs | AJaeger, even if they are executed on several jenkins systems, they should be collated on https://jenkins07.openstack.org/job/gate-tempest-dsvm-full-glusterfs/ | 09:16 |
| AJaeger | NO | 09:16 |
| AJaeger | they are collected on logs.openstack.org | 09:16 |
| deepakcs | AJaeger, this means that jenkins doesn't see the 2 check experimetnals as 2 jobs, it still 1 job, but 2 runs, maybe ? | 09:17 |
| AJaeger | the other job might have been executed by jenkins01 or 02...06 | 09:17 |
| *** derekh has joined #openstack-infra | 09:22 | |
| deepakcs | AJaeger, 01..07 are just mirrors/HA , not different slaves | 09:24 |
| AJaeger | deepakcs: they are different slaves - not mirrors/HA | 09:25 |
| AJaeger | each runs independendently and zuul distributes the jobs to them | 09:25 |
| *** dizquierdo has quit IRC | 09:25 | |
| AJaeger | so, a job run on jenkins02 will not be seen via jenkins07 | 09:25 |
| AJaeger | Therefore, don't check jenkins, use the links from review.openstack.org | 09:26 |
| BobBall | Any idea why I get AttributeError: 'NoneType' object has no attribute 'name' | 09:30 |
| BobBall | errors from Zuul? | 09:30 |
| BobBall | http://paste.openstack.org/show/182908/ | 09:30 |
| deepakcs | AJaeger, i don't think thats correct, i jsut saw https://jenkins06.openstack.org/job/gate-tempest-dsvm-full-glusterfs/ - it lists the same job that was on o7 | 09:32 |
| deepakcs | AJaeger, u r right... my bad | 09:33 |
| *** esker has joined #openstack-infra | 09:33 | |
| deepakcs | AJaeger, so there is no way to figure which jenkins {01..07} the jobs are scheduled ? | 09:33 |
| AJaeger | deepakcs: the same job definition | 09:34 |
| deepakcs | AJaeger, for eg: i forgot the patch URL, just wanted to monitor all the jobs spawned, I need to check jenkins {01..07} manually ? | 09:34 |
| AJaeger | But not the same change | 09:34 |
| *** ildikov has joined #openstack-infra | 09:34 | |
| *** ihrachyshka has joined #openstack-infra | 09:34 | |
| AJaeger | So, you can check status.openstack.org/zuul to monitor when jobs get started and see on which jenkins | 09:34 |
| AJaeger | But this is non-deterministic, depending on load of the jenkins systems | 09:35 |
| *** abhishek_cb has joined #openstack-infra | 09:35 | |
| AJaeger | and afterwwards, you go to your change on review.openstack.org and click on the experimental link to get the log files | 09:35 |
| deepakcs | AJaeger, thanks, get it now | 09:36 |
| abhishek_cb | hi, can anyone tell me that using this command will help me running tempest for cinder or not "sudo -H -u tempest tox -e all -- volume" | 09:36 |
| *** armax has joined #openstack-infra | 09:43 | |
| *** otter768 has joined #openstack-infra | 09:44 | |
| *** belmoreira has joined #openstack-infra | 09:47 | |
| BobBall | How does zuul get installed through puppet? I'm trying to find out why I get version 2.0.0.post351 and if there is a newer version that'll fix my issues (seems that the NoneType issue is linked to cross-project dependencies) | 09:48 |
| *** otter768 has quit IRC | 09:49 | |
| deepakcs | abhishek_cb, https://wiki.openstack.org/wiki/Cinder/tested-3rdParty-drivers#What_tests_do_I_use.3F | 09:51 |
| deepakcs | abhishek_cb, and this too: https://ask.openstack.org/en/question/8778/using-tempest-for-testing-only-keystone-and-cinder/ | 09:52 |
| deepakcs | abhishek_cb, the cmd you gave might work too.. what happens when u run it ? I never tried that, but it does looks logical | 09:52 |
| *** ruhe has quit IRC | 09:53 | |
| *** SergeyLukjanov has quit IRC | 09:53 | |
| *** DinaBelova has quit IRC | 09:53 | |
| *** alazarev has quit IRC | 09:53 | |
| *** NikitaKonovalov has quit IRC | 09:53 | |
| *** dmitryme has quit IRC | 09:53 | |
| abhishek_cb | deepakcs : that i have to see, i figured it out fro sos-ci beacuse they are using this command to run tempest | 09:53 |
| *** DinaBelova has joined #openstack-infra | 09:53 | |
| deepakcs | abhishek_cb, then you should just run adn see ;) | 09:54 |
| *** dmitryme has joined #openstack-infra | 09:54 | |
| *** alazarev has joined #openstack-infra | 09:54 | |
| *** NikitaKonovalov has joined #openstack-infra | 09:54 | |
| *** ruhe has joined #openstack-infra | 09:54 | |
| *** SergeyLukjanov has joined #openstack-infra | 09:55 | |
| *** ddieterly has joined #openstack-infra | 10:03 | |
| *** ssam2 has joined #openstack-infra | 10:04 | |
| *** ddieterly has quit IRC | 10:07 | |
| *** jedimike has joined #openstack-infra | 10:07 | |
| *** rkukura_ has joined #openstack-infra | 10:19 | |
| *** e0ne has joined #openstack-infra | 10:19 | |
| *** vdo has joined #openstack-infra | 10:20 | |
| *** skolekonov has joined #openstack-infra | 10:20 | |
| openstackgerrit | Aleksey Ripinen proposed openstack-infra/python-storyboardclient: Added tags controller https://review.openstack.org/157857 | 10:21 |
| *** rkukura has quit IRC | 10:21 | |
| *** rkukura_ is now known as rkukura | 10:21 | |
| *** pelix has joined #openstack-infra | 10:30 | |
| *** katyafervent is now known as katyafervent_awa | 10:37 | |
| abhishek_cb | deepakcs : that command is running but with a minor change "sudo -H -u tempest tox -eall -- volume" | 10:38 |
| deepakcs | abhishek_cb, good to know | 10:38 |
| *** katyafervent_awa is now known as katyafervent | 10:41 | |
| *** andreykurilin_ has quit IRC | 10:49 | |
| *** koolhead17 has quit IRC | 10:52 | |
| *** claudiub has joined #openstack-infra | 10:54 | |
| *** tosky has joined #openstack-infra | 11:01 | |
| *** ddieterly has joined #openstack-infra | 11:04 | |
| *** ddieterly has quit IRC | 11:08 | |
| *** pc_m has joined #openstack-infra | 11:09 | |
| *** markvoelker has joined #openstack-infra | 11:10 | |
| *** dizquierdo has joined #openstack-infra | 11:14 | |
| openstackgerrit | Aleksey Ripinen proposed openstack-infra/python-storyboardclient: Added branches controller https://review.openstack.org/159792 | 11:15 |
| *** markvoelker has quit IRC | 11:17 | |
| openstackgerrit | Aleksey Ripinen proposed openstack-infra/infra-specs: Foreign keys support in StoryBoard https://review.openstack.org/159406 | 11:21 |
| *** abhishek_cb has quit IRC | 11:28 | |
| *** yamamoto has quit IRC | 11:29 | |
| *** coolsvap is now known as coolsvap_ | 11:30 | |
| *** mpaolino has quit IRC | 11:31 | |
| *** mpaolino has joined #openstack-infra | 11:31 | |
| *** lxsli is now known as thelexx | 11:31 | |
| *** e0ne has quit IRC | 11:33 | |
| *** deepakcs has quit IRC | 11:33 | |
| *** ldnunes has joined #openstack-infra | 11:35 | |
| *** rlandy has joined #openstack-infra | 11:36 | |
| *** e0ne has joined #openstack-infra | 11:36 | |
| *** mpaolino has quit IRC | 11:36 | |
| *** _shaps_ has joined #openstack-infra | 11:36 | |
| *** mpaolino has joined #openstack-infra | 11:36 | |
| *** _shaps_ has quit IRC | 11:36 | |
| *** _shaps_ has joined #openstack-infra | 11:37 | |
| *** mpaolino has quit IRC | 11:38 | |
| *** mpaolino has joined #openstack-infra | 11:38 | |
| *** mpaolino has quit IRC | 11:40 | |
| *** mpaolino has joined #openstack-infra | 11:41 | |
| *** otter768 has joined #openstack-infra | 11:45 | |
| openstackgerrit | Aleksey Ripinen proposed openstack-infra/storyboard: Remove is_active from task model https://review.openstack.org/159135 | 11:46 |
| *** dtantsur|bbl is now known as dtantsur | 11:49 | |
| *** otter768 has quit IRC | 11:50 | |
| *** ghostpl_ has joined #openstack-infra | 11:54 | |
| *** ghostpl_ has quit IRC | 11:54 | |
| *** ghostpl_ has joined #openstack-infra | 11:55 | |
| *** ghostpl_ has joined #openstack-infra | 11:56 | |
| *** cdent has joined #openstack-infra | 11:57 | |
| *** coolsvap_ is now known as coolsvap | 12:02 | |
| openstackgerrit | Sergey Lukjanov proposed openstack-infra/jenkins-job-builder: Add slack notification support, largely inspired by hipchat module https://review.openstack.org/136627 | 12:03 |
| *** afazekas has joined #openstack-infra | 12:04 | |
| *** dims has joined #openstack-infra | 12:04 | |
| *** e0ne is now known as e0ne_ | 12:04 | |
| *** ddieterly has joined #openstack-infra | 12:04 | |
| *** rfolco has joined #openstack-infra | 12:07 | |
| *** ddieterly has quit IRC | 12:09 | |
| *** markvoelker has joined #openstack-infra | 12:13 | |
| *** yamamoto has joined #openstack-infra | 12:14 | |
| *** ddieterly has joined #openstack-infra | 12:14 | |
| *** cody-somerville has joined #openstack-infra | 12:16 | |
| *** yolanda has quit IRC | 12:17 | |
| *** markvoelker has quit IRC | 12:17 | |
| *** dmsimard_away is now known as dmsimard | 12:17 | |
| *** ghostpl_ has quit IRC | 12:19 | |
| *** amotoki has joined #openstack-infra | 12:21 | |
| *** spredzy is now known as spredzy|afk | 12:22 | |
| *** ghostpl_ has joined #openstack-infra | 12:22 | |
| *** aysyd has joined #openstack-infra | 12:23 | |
| *** ghostpl_ has quit IRC | 12:27 | |
| *** ghostpl_ has joined #openstack-infra | 12:29 | |
| *** teran has joined #openstack-infra | 12:32 | |
| *** abhishek_cb has joined #openstack-infra | 12:34 | |
| openstackgerrit | Merged openstack-infra/project-config: Add rally job to ec2-api project https://review.openstack.org/159138 | 12:36 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config: Cleanup accessbot and gerritbot https://review.openstack.org/159585 | 12:36 |
| abhishek_cb | hi all, after following ramy's guide i have setup the Ci environment but it is not able to read gerrit events can anyone help me with that?? | 12:37 |
| AJaeger | abhishek_cb: here's the current information: http://ci.openstack.org/third_party.html | 12:38 |
| AJaeger | AFAIK that's updated since ramy's guide was written | 12:39 |
| AJaeger | I can't help further. | 12:39 |
| *** abhishek_cb has quit IRC | 12:43 | |
| *** mpaolino has quit IRC | 12:48 | |
| *** teran_ has joined #openstack-infra | 12:48 | |
| *** mpaolino has joined #openstack-infra | 12:48 | |
| *** ghostpl_ has quit IRC | 12:48 | |
| *** dtantsur is now known as dtantsur|lunch | 12:49 | |
| *** tonytan4ever has joined #openstack-infra | 12:49 | |
| *** mpaolino has quit IRC | 12:50 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config: Cleanup accessbot and gerritbot https://review.openstack.org/159585 | 12:50 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config: Sort accessbot/channels.yaml https://review.openstack.org/159818 | 12:50 |
| *** mpaolino has joined #openstack-infra | 12:50 | |
| *** teran_ has quit IRC | 12:51 | |
| *** teran has quit IRC | 12:52 | |
| *** ZZelle has quit IRC | 12:52 | |
| *** weshay has joined #openstack-infra | 12:52 | |
| *** ZZelle has joined #openstack-infra | 12:52 | |
| *** mpaolino has quit IRC | 12:55 | |
| *** mpaolino has joined #openstack-infra | 12:56 | |
| *** woodster_ has joined #openstack-infra | 12:56 | |
| *** mpaolino has quit IRC | 12:58 | |
| *** mpaolino has joined #openstack-infra | 12:58 | |
| *** e0ne_ is now known as e0ne | 12:59 | |
| *** rfolco has quit IRC | 13:04 | |
| *** markvoelker has joined #openstack-infra | 13:14 | |
| *** teran has joined #openstack-infra | 13:15 | |
| *** abhirc has joined #openstack-infra | 13:15 | |
| *** rfolco has joined #openstack-infra | 13:17 | |
| *** markvoelker has quit IRC | 13:19 | |
| openstackgerrit | Julia Kreger proposed openstack-infra/shade: Add behavior to enable ironic noauth mode https://review.openstack.org/159609 | 13:19 |
| *** mbacchi has joined #openstack-infra | 13:19 | |
| *** mpaolino has quit IRC | 13:19 | |
| *** baoli has joined #openstack-infra | 13:19 | |
| *** mpaolino has joined #openstack-infra | 13:20 | |
| *** baoli has quit IRC | 13:24 | |
| *** yolanda has joined #openstack-infra | 13:25 | |
| *** scheuran has quit IRC | 13:25 | |
| *** baoli has joined #openstack-infra | 13:25 | |
| *** esker has quit IRC | 13:25 | |
| *** bswartz has quit IRC | 13:25 | |
| *** armax has quit IRC | 13:26 | |
| *** kgiusti has joined #openstack-infra | 13:26 | |
| *** dims is now known as dimsum__ | 13:27 | |
| *** dtantsur|lunch is now known as dtantsur | 13:29 | |
| *** ghostpl_ has joined #openstack-infra | 13:32 | |
| *** ghostpl_ has quit IRC | 13:37 | |
| *** gordc has joined #openstack-infra | 13:37 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config: Sort gerritbot/channels.yaml https://review.openstack.org/159838 | 13:40 |
| ctlaugh | I've got nodepool running, and it is successfully building an image. It is, however, failing when it tries to launch a new VM and ssh into it. I can see error messages in the debug output saying login failed due to a bad key. I have tried manually ssh into a running VM (ssh ubuntu@vm-ip-addr) using the key in nodepool's ~/.ssh/id_rsa and it works. Any ideas? Possibly another config option I've missed somewhere...? | 13:41 |
| BobBall | no ctlaugh | 13:41 |
| BobBall | this is a frustration | 13:41 |
| BobBall | the docs I was following were wrong too | 13:41 |
| BobBall | :) | 13:42 |
| *** ddieterly has quit IRC | 13:42 | |
| BobBall | you need the env var NODEPOOL_SSH_KEY setup in the environment you run nodepoold from | 13:42 |
| BobBall | that's present in /etc/default/nodepool so should be there if you're using the init scripts, but for some reason they don't always work. | 13:42 |
| BobBall | Also, if you set up your env using os-ext-testing then the key will have spaces in, which isn't allowed | 13:42 |
| BobBall | https://github.com/citrix-openstack/os-ext-testing/commit/25c42e9edbfb042ca516cef78178c6bb135123fe is what I think you need | 13:43 |
| *** otter768 has joined #openstack-infra | 13:46 | |
| ctlaugh | BobBall: Yes - I setup using os-ext-testing, and, following the instructions, was launching nodepool using nodepoold -d to test it. That explains why the env var isn't getting set. | 13:47 |
| ctlaugh | BobBall: Thank you so much | 13:47 |
| BobBall | d'oh - typo in those instructions - should be sourcing /etc/default/nodepool | 13:48 |
| ctlaugh | BobBall: :) | 13:48 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config: Sort gerritbot/channels.yaml https://review.openstack.org/159838 | 13:48 |
| pc_m | I have some questions on operating system support in tests and devstack, anyone able to answer? | 13:49 |
| ctlaugh | BobBall: I was going to ask this later, but how do you get nodepool to start automatically? The instructions show launching manually, but that can't be done long-term..... | 13:49 |
| pc_m | Wondering if devstack works with only ubuntu 14.04, or will it work with 14.10? | 13:49 |
| BobBall | No idea ctlaugh - seems the init scripts don't work. I did want to look at that a while ago, but other priorities have pulled me away :) | 13:50 |
| BobBall | I'm just running it manually in a screen session | 13:50 |
| *** otter768 has quit IRC | 13:51 | |
| *** jerryz has quit IRC | 13:52 | |
| ctlaugh | BobBall: I haven't gotten that far either, yet. Thanks for the help | 13:53 |
| *** abhirc has quit IRC | 13:53 | |
| ctlaugh | BobBall: ah - one more ?.... My key in NODEPOOL_SSH_KEY has spaces in it, i.e. ='ssh-rsa AAAAblahblah.. ubuntu@server' is that ok? I saw various instructions saying that the key needed to have no spaces, and wasn't sure if I made another mistake somewhere. | 13:54 |
| BobBall | That's another bug in os-ext-testing. I said that too :) Also fixed in the commit I referenced | 13:55 |
| openstackgerrit | Nikolay Fedotov proposed openstack-infra/project-config: Functional test job for Kolla https://review.openstack.org/158726 | 13:56 |
| *** mfink_ has joined #openstack-infra | 13:56 | |
| ctlaugh | BobBall: Sorry, I didn't scroll down far enough... I only looked at the change in README.md. Thank you! | 13:57 |
| BobBall | no probs :) | 13:57 |
| *** mpaolino has quit IRC | 13:58 | |
| *** radez_g0n3 is now known as radez | 13:58 | |
| *** mpaolino has joined #openstack-infra | 13:58 | |
| *** jcoufal_ has joined #openstack-infra | 13:59 | |
| *** mpaolino has quit IRC | 14:00 | |
| *** mpaolino has joined #openstack-infra | 14:00 | |
| *** mfink_ has quit IRC | 14:01 | |
| *** mfink_ has joined #openstack-infra | 14:01 | |
| *** bswartz has joined #openstack-infra | 14:01 | |
| *** abhirc has joined #openstack-infra | 14:02 | |
| openstackgerrit | Julia Kreger proposed openstack-infra/shade: Add behavior to enable ironic noauth mode https://review.openstack.org/159609 | 14:02 |
| *** jcoufal has quit IRC | 14:02 | |
| *** abhirc has quit IRC | 14:04 | |
| *** mtanino has quit IRC | 14:07 | |
| *** mattfarina has joined #openstack-infra | 14:08 | |
| *** amitgandhinz has joined #openstack-infra | 14:09 | |
| *** amitgandhinz has quit IRC | 14:10 | |
| *** amitgandhinz has joined #openstack-infra | 14:10 | |
| *** jcoufal_ has quit IRC | 14:12 | |
| *** jcoufal has joined #openstack-infra | 14:12 | |
| *** markvoelker has joined #openstack-infra | 14:15 | |
| anteaya | jeblair: ack | 14:15 |
| *** bhunter71 has joined #openstack-infra | 14:16 | |
| *** markvoelker has quit IRC | 14:20 | |
| *** markvoelker has joined #openstack-infra | 14:22 | |
| *** mbacchi has quit IRC | 14:22 | |
| *** ddieterly has joined #openstack-infra | 14:22 | |
| *** amotoki has quit IRC | 14:24 | |
| *** mestery has quit IRC | 14:25 | |
| *** tiswanso has joined #openstack-infra | 14:25 | |
| *** mbacchi has joined #openstack-infra | 14:26 | |
| *** mriedem has joined #openstack-infra | 14:28 | |
| *** rlandy has quit IRC | 14:29 | |
| *** dkliban_afk is now known as dkliban | 14:31 | |
| *** coolsvap is now known as coolsvap_ | 14:31 | |
| *** sabeen1 has joined #openstack-infra | 14:33 | |
| *** dimsum__ has quit IRC | 14:36 | |
| *** tgohad has quit IRC | 14:38 | |
| *** dims_ has joined #openstack-infra | 14:38 | |
| *** claudiub has quit IRC | 14:39 | |
| *** psedlak has quit IRC | 14:39 | |
| mkj_wk | ZZelle: thanks for the detailed review yesterday. Very helpful, much improved as a result. | 14:40 |
| ZZelle | mkj_wk, :) | 14:40 |
| *** mestery has joined #openstack-infra | 14:41 | |
| *** bknudson has quit IRC | 14:42 | |
| mkj_wk | take a look at the test case that I put on a diet, and see if you think it can squeeze in as it is | 14:42 |
| mkj_wk | I can easily split it in two, but the part that tests the first half will invoke the original test because everything in the original test is required to set up for the second half | 14:42 |
| *** e0ne has quit IRC | 14:42 | |
| mkj_wk | so it would be duplicate work, and it's already two 15-20 second test cases, most of which is in the setup/teardown | 14:43 |
| *** e0ne has joined #openstack-infra | 14:43 | |
| mkj_wk | and I hate to make tests unnecessarily longer because I've seen it lead to people skimping on running tests before submitting code | 14:44 |
| *** psedlak has joined #openstack-infra | 14:44 | |
| openstackgerrit | Jaume Devesa proposed openstack-infra/project-config: Add non-voting tempest gate for Midonet https://review.openstack.org/159867 | 14:44 |
| mkj_wk | with that in mind, I'll still split it apart that way if you feel that the slimmed-down test case (with the worst of the "how" bits removed) still needs to be broken up, as you see it | 14:45 |
| *** esker has joined #openstack-infra | 14:47 | |
| *** sabeen1 has quit IRC | 14:47 | |
| openstackgerrit | Matt Riedemann proposed openstack-infra/elastic-recheck: Add query for neutron ipv6 bug 1426397 https://review.openstack.org/159868 | 14:47 |
| openstack | bug 1426397 in neutron "tempest.scenario.test_network_v6.TestGettingAddress.test_slaac_from_os intermittently fails with MismatchError" [Undecided,New] https://launchpad.net/bugs/1426397 | 14:47 |
| *** dansmith is now known as superdan | 14:48 | |
| *** mrmartin has quit IRC | 14:50 | |
| *** sigmavirus24_awa is now known as sigmavirus24 | 14:50 | |
| *** cody-somerville has quit IRC | 14:51 | |
| anteaya | pc_m: we test on precise and trusty | 14:51 |
| *** psedlak has quit IRC | 14:52 | |
| anteaya | pc_m: we only use ubuntu versions which are long term support versions | 14:52 |
| anteaya | it takes us too long to set things up to use any of the versions in between and they aren't supported long enough for our needs | 14:52 |
| *** mtanino has joined #openstack-infra | 14:52 | |
| sdague | pc_m: devstack isn't testing all the time on 14.10, but it probably works | 14:53 |
| *** psedlak has joined #openstack-infra | 14:54 | |
| pc_m | sdague: anteaya: Thanks. The issue was that it looks like we would need two different commands for 14.04 and 14.10, due to an issue. | 14:55 |
| pc_m | was concerned about how to support that difference. Seems like we don't have to worry about it. | 14:55 |
| sdague | pc_m: well, the patch is welcomed in devstack | 14:56 |
| sdague | we have conditional ability there if you want to propose it | 14:57 |
| pc_m | Looks like there is currently an issue, where an apparmor command is different in 14.10 and they have a request to make it the same. | 14:57 |
| *** mpaolino has quit IRC | 14:58 | |
| openstackgerrit | Cedric Brandily proposed openstack-infra/git-review: Support authentication in run_http_exc https://review.openstack.org/124410 | 14:58 |
| openstackgerrit | Cedric Brandily proposed openstack-infra/git-review: Taz https://review.openstack.org/159875 | 14:58 |
| *** mpaolino has joined #openstack-infra | 14:58 | |
| *** unicell1 has joined #openstack-infra | 15:00 | |
| *** e0ne is now known as e0ne_ | 15:01 | |
| *** mpaolino has quit IRC | 15:02 | |
| *** unicell has quit IRC | 15:02 | |
| *** mpaolino has joined #openstack-infra | 15:03 | |
| *** mpaolino has quit IRC | 15:06 | |
| *** mpaolino has joined #openstack-infra | 15:06 | |
| *** ildikov has quit IRC | 15:06 | |
| *** nelsnelson has joined #openstack-infra | 15:08 | |
| jedimike | what version of git is running on the server that runs manage-projects? we're getting hit by this http://article.gmane.org/gmane.comp.version-control.git/264493 which was fixed in the 1.8 series, but precise comes with 1.7, so are you running trusty on the gerrit server? or using git from a ppa? | 15:08 |
| *** MarkAtwood has joined #openstack-infra | 15:08 | |
| *** yamamoto has quit IRC | 15:09 | |
| *** sdake_ has joined #openstack-infra | 15:09 | |
| openstackgerrit | Jaume Devesa proposed openstack-infra/project-config: Add non-voting tempest gate for Midonet https://review.openstack.org/159867 | 15:10 |
| *** sdake__ has joined #openstack-infra | 15:10 | |
| openstackgerrit | Merged openstack-infra/elastic-recheck: Add query for neutron ipv6 bug 1426397 https://review.openstack.org/159868 | 15:12 |
| *** zz_jgrimm is now known as jgrimm | 15:12 | |
| openstack | bug 1426397 in neutron "tempest.scenario.test_network_v6.TestGettingAddress.test_slaac_from_os intermittently fails with MismatchError" [Undecided,New] https://launchpad.net/bugs/1426397 - Assigned to Henry Gessau (gessau) | 15:12 |
| pc_m | sdague: Given the time crunch for K-3, probably want to just support 14.04 right now. Can follow up later with conditional if it makes sense to do that for a non-LTE release. | 15:12 |
| anteaya | jeblair: I hope I am understanding your conversation with marun from last night correctly, in that you have been operating under the belief that devstack-gate check stable branch jobs should be non-voting and aren't | 15:12 |
| anteaya | jeblair: I've take a look at past patches and I see the check jobs to devstack-gate changed around Oct. 8 - 15 | 15:13 |
| anteaya | a merged patch from Oct. 8 showing devstack-gate check jobs: https://review.openstack.org/#/c/124190/ | 15:14 |
| *** sdake_ has quit IRC | 15:14 | |
| anteaya | a merged patch showing check jobs from Oct. 14 and 15: https://review.openstack.org/#/c/126710/ | 15:14 |
| anteaya | from what I see stable branch check jobs were never non-voting | 15:15 |
| openstackgerrit | Cedric Brandily proposed openstack-infra/git-review: Support authentication in run_http_exc https://review.openstack.org/124410 | 15:18 |
| openstackgerrit | Andrey Pavlov proposed openstack-infra/project-config: Create our own rally job for ec2-api https://review.openstack.org/159884 | 15:19 |
| *** carl_baldwin has joined #openstack-infra | 15:19 | |
| *** teran_ has joined #openstack-infra | 15:20 | |
| jeblair | anteaya: https://review.openstack.org/#/c/83076/ | 15:20 |
| *** teran__ has joined #openstack-infra | 15:22 | |
| *** teran has quit IRC | 15:22 | |
| *** yamamoto has joined #openstack-infra | 15:23 | |
| *** teran_ has quit IRC | 15:23 | |
| *** ChuckC has quit IRC | 15:23 | |
| *** spredzy|afk is now known as spredzy | 15:26 | |
| *** teran__ has quit IRC | 15:26 | |
| anteaya | ah sorry I didn't go back far enough | 15:26 |
| anteaya | I will continue to dig | 15:26 |
| *** teran has joined #openstack-infra | 15:27 | |
| zxiiro | With Jenkins Job Builder, does it support switching job types for existing jobs? or do I have to delete a job first? | 15:29 |
| zxiiro | for example if I wanted to change a Maven Project to a Matrix Project | 15:29 |
| *** yamamoto has quit IRC | 15:29 | |
| anteaya | jeblair: they also went through a period of time where stable branches didnt' test devstack-gate in check | 15:30 |
| *** achanda has joined #openstack-infra | 15:31 | |
| anteaya | https://review.openstack.org/#/c/122727/ | 15:31 |
| jeblair | anteaya: i'm not sure the archeology is as important as just fixing them :) | 15:31 |
| *** annegentle has joined #openstack-infra | 15:31 | |
| *** e0ne_ is now known as e0ne | 15:31 | |
| jeblair | zxiiro: i believe for some job type like that, you have to delete | 15:32 |
| *** markvoelker has quit IRC | 15:32 | |
| jeblair | zxiiro: at least i know it's the case for maven->freestyle | 15:32 |
| zxiiro | that's what i figured, thanks for confirming | 15:32 |
| *** markvoelker has joined #openstack-infra | 15:32 | |
| anteaya | between sept. 4 https://review.openstack.org/#/c/116995/ and sept. 6: https://review.openstack.org/#/c/113658/ | 15:33 |
| anteaya | oh I thought you said you wanted to know what happened | 15:33 |
| *** prad has joined #openstack-infra | 15:34 | |
| *** BobBall is now known as BobBall_AWOL | 15:34 | |
| *** mpaolino has quit IRC | 15:36 | |
| *** mrunge has quit IRC | 15:36 | |
| *** teran has quit IRC | 15:36 | |
| *** achanda has quit IRC | 15:36 | |
| *** mpaolino has joined #openstack-infra | 15:36 | |
| *** teran has joined #openstack-infra | 15:36 | |
| *** markvoelker has quit IRC | 15:37 | |
| *** teran_ has joined #openstack-infra | 15:37 | |
| *** teran_ has quit IRC | 15:37 | |
| *** teran_ has joined #openstack-infra | 15:37 | |
| *** abhirc has joined #openstack-infra | 15:39 | |
| *** amotoki has joined #openstack-infra | 15:40 | |
| *** abhirc has quit IRC | 15:40 | |
| *** salv-orlando has quit IRC | 15:40 | |
| *** teran has quit IRC | 15:41 | |
| *** stevemar has joined #openstack-infra | 15:44 | |
| *** nelsnelson has quit IRC | 15:45 | |
| *** tsg has joined #openstack-infra | 15:45 | |
| *** prad has quit IRC | 15:46 | |
| *** otter768 has joined #openstack-infra | 15:47 | |
| *** stevemar2 has joined #openstack-infra | 15:47 | |
| *** stevemar has quit IRC | 15:47 | |
| *** abhirc has joined #openstack-infra | 15:49 | |
| *** afazekas has quit IRC | 15:50 | |
| openstackgerrit | James E. Blair proposed openstack-infra/gear: Move running check in getJob https://review.openstack.org/159899 | 15:50 |
| *** bknudson has joined #openstack-infra | 15:52 | |
| *** otter768 has quit IRC | 15:52 | |
| openstackgerrit | Merged openstack-infra/gear: Fix race between stopWaitingForJobs() and getJob() https://review.openstack.org/137664 | 15:52 |
| *** bknudson has quit IRC | 15:53 | |
| *** armax has joined #openstack-infra | 15:53 | |
| *** armax has quit IRC | 15:53 | |
| openstackgerrit | Anita Kuno proposed openstack-infra/project-config: Make devstack-gate stable jobs non-voting https://review.openstack.org/159900 | 15:54 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config: Add docs to the magnum gate https://review.openstack.org/149334 | 15:54 |
| *** armax has joined #openstack-infra | 15:54 | |
| openstackgerrit | Oleg Gashev proposed openstack-dev/pbr: Update requirements to fix "python setup.py --help-commands" call. https://review.openstack.org/159901 | 15:54 |
| annegentle | too early for reed :) Anyone know when the next batch of ATC badges will be run? | 15:55 |
| anteaya | annegentle: I do not no | 15:55 |
| openstackgerrit | Nikolay Fedotov proposed openstack-infra/project-config: Functional test job for Kolla https://review.openstack.org/158726 | 15:56 |
| jgriffith | mordred: BTW, my FAVORITE github README is still and likely always will be the one you put together for Shade | 15:56 |
| AJaeger | annegentle: I remember something about Kilo3 | 15:56 |
| openstackgerrit | Anita Kuno proposed openstack-infra/project-config: Make devstack-gate stable jobs non-voting https://review.openstack.org/159900 | 15:56 |
| jgriffith | mordred: "hipster, narcissitic...." | 15:56 |
| jgriffith | just the best | 15:56 |
| rcarrillocruz | jgriffith: i had the same feeling the first time i read it :D | 15:57 |
| jgriffith | rcarrillocruz: :) | 15:57 |
| *** markvoelker has joined #openstack-infra | 15:58 | |
| *** bswartz has quit IRC | 15:59 | |
| *** primemin1sterp is now known as primeministerp | 16:00 | |
| mordred | jgriffith: :) | 16:00 |
| mordred | jgriffith: glad you're enjoying it | 16:00 |
| annegentle | a-okay thanks y'all | 16:01 |
| anteaya | jeblair mtreinish I think I might have found the issue, devstack-gate and tempest use the same job names for the stable branch testing, devstack-gate wants them non-voting and I'm going with tempest wants them voting? | 16:01 |
| jeblair | anteaya: i believe that's the case | 16:02 |
| mtreinish | anteaya: yes if they're not voting on tempest we will break everyone all the time | 16:02 |
| anteaya | okay you two decide | 16:02 |
| mtreinish | which honestly I'd be fine with as long as I don't have to fix the stable branches | 16:02 |
| anteaya | let me know what you come up with and I will compose patches | 16:02 |
| jgriffith | mtreinish: BTW https://review.openstack.org/#/c/159713/ | 16:02 |
| jeblair | anteaya: devstack-gate needs -nv jobs | 16:02 |
| *** spzala has joined #openstack-infra | 16:02 | |
| anteaya | that is an easy fix | 16:03 |
| anteaya | I'll do that | 16:03 |
| Shrews | jgriffith: mordred: it's both funny and sad | 16:03 |
| mtreinish | jgriffith: yep I saw that and already poked sdague and mriedem about it :) | 16:03 |
| jgriffith | coolio | 16:03 |
| mriedem | it's got it's own tab, warm and safe | 16:04 |
| mriedem | rolling through some other reviews first | 16:04 |
| *** mpaolino has quit IRC | 16:04 | |
| mriedem | jgriffith: shall i ping jungleboyj to ding you on the typo in the commit message :) | 16:04 |
| *** mpaolino has joined #openstack-infra | 16:04 | |
| *** mrmartin has joined #openstack-infra | 16:05 | |
| *** AJaeger has quit IRC | 16:05 | |
| mordred | Shrews: btw - we should also plumb the insecure, verify and cert args to all of the shade client constructors | 16:06 |
| *** bknudson has joined #openstack-infra | 16:06 | |
| *** teran_ has quit IRC | 16:06 | |
| mordred | Shrews: and then add all three of those as params to the central openstack_full_argument_spec thing in the ansible repo - they are valid arguments for _everything_ | 16:06 |
| *** teran has joined #openstack-infra | 16:07 | |
| *** mpavone has quit IRC | 16:07 | |
| Shrews | mordred: We need to start bug tracking or something. Too many things we "need to do" to remember them all. | 16:07 |
| Shrews | Would also help people who want to get involved | 16:08 |
| mordred | Shrews: ++ | 16:08 |
| Shrews | mordred: and then we can assign them all to SpamapS | 16:08 |
| mordred | Shrews: I like the way you think sir | 16:08 |
| mordred | Shrews: so you're saying my etherpad is _not_ the best way to handle this | 16:09 |
| Shrews | mordred: sadly, no | 16:09 |
| * mordred sadlys | 16:09 | |
| Shrews | etherpads require bookmarking b/c who the heck can remember them or google search for them | 16:10 |
| * Shrews stabs etherpad with an etherspoon | 16:10 | |
| *** dangers_away is now known as dangers | 16:12 | |
| anteaya | jeblair: the default for -nv jobs is being off for stable: http://git.openstack.org/cgit/openstack-infra/project-config/tree/zuul/layout.yaml#n552 | 16:12 |
| anteaya | so I'm writing out specific rules for the devstack-gate jobs? | 16:13 |
| *** ChuckC has joined #openstack-infra | 16:16 | |
| *** ChuckC has quit IRC | 16:17 | |
| *** ChuckC has joined #openstack-infra | 16:17 | |
| *** david-lyle_afk is now known as david-lyle | 16:19 | |
| asselin | anyone getting this error stacking on master? "Routes is in an unsupported or invalid wheel" http://15.126.198.151/41/157441/5/check/3par-fc-driver-master-client-pip-eos10-dsvm/b2c0627/logs/devstacklog.txt.gz | 16:20 |
| *** jamesmcarthur has joined #openstack-infra | 16:20 | |
| *** ChuckC has quit IRC | 16:20 | |
| *** tonytan4ever has quit IRC | 16:20 | |
| *** tonytan4ever has joined #openstack-infra | 16:21 | |
| *** tnovacik has quit IRC | 16:21 | |
| *** ChuckC has joined #openstack-infra | 16:21 | |
| jeblair | anteaya: i guess so. i'm not entirely sure why that rule should be the case though. | 16:22 |
| jeblair | anteaya: i worry that it's based on a particular situtation from a while ago (like ironic-nv jobs on nova or something), and i don't know whethere it is still needed. it doesn't seem a very future-proof rule :( | 16:23 |
| jeblair | anteaya: i guess we would need an inventory of -nv jobs to find out whether the balance has shifted | 16:23 |
| *** dannywilson has joined #openstack-infra | 16:24 | |
| *** tnovacik has joined #openstack-infra | 16:24 | |
| *** markvoelker has quit IRC | 16:25 | |
| openstackgerrit | Anita Kuno proposed openstack-infra/project-config: Make devstack-gate stable jobs non-voting https://review.openstack.org/159900 | 16:25 |
| *** tonytan4ever has quit IRC | 16:25 | |
| anteaya | jeblair: I put the history in my commit message | 16:26 |
| anteaya | let me know what you think | 16:26 |
| anteaya | the history I found anyway | 16:26 |
| *** reed has joined #openstack-infra | 16:26 | |
| anteaya | I'll see what we currenly have by way of -nv jobs in zuul/layout.yaml | 16:27 |
| *** mpaolino has quit IRC | 16:28 | |
| *** zz_dimtruck is now known as dimtruck | 16:29 | |
| *** mpaolino has joined #openstack-infra | 16:29 | |
| anteaya | one job: check-tempest-dsvm-ironic-parallel-nv currently in ironic's check pipeline and tempest's experimental pipeline | 16:29 |
| *** prad has joined #openstack-infra | 16:30 | |
| *** yamamoto has joined #openstack-infra | 16:30 | |
| jeblair | anteaya: it may be worth specifying that job as specifically excluding stable branches and then removing the stable exclusion from *-nv | 16:30 |
| *** MarkAtwood has quit IRC | 16:31 | |
| anteaya | oh | 16:32 |
| *** tnovacik has quit IRC | 16:32 | |
| anteaya | I can try that | 16:32 |
| *** mpaolino has quit IRC | 16:32 | |
| jeblair | if it's only one job, i think that will make things more clear and reduce future confusion | 16:32 |
| *** markvoelker has joined #openstack-infra | 16:32 | |
| *** mpaolino has joined #openstack-infra | 16:32 | |
| *** stevemar3 has joined #openstack-infra | 16:32 | |
| *** jcoufal has quit IRC | 16:33 | |
| anteaya | I'll offer that is a parent to my change | 16:33 |
| *** stevemar2 has quit IRC | 16:33 | |
| *** markus_z has quit IRC | 16:35 | |
| *** yamamoto has quit IRC | 16:35 | |
| *** david-lyle is now known as david-lyle_Afk | 16:35 | |
| *** david-lyle_Afk is now known as david-lyle_afk | 16:35 | |
| *** vilobhmm has joined #openstack-infra | 16:37 | |
| *** bnemec is now known as beekneemech | 16:37 | |
| *** marcusvrn1 has joined #openstack-infra | 16:38 | |
| *** rwsu-afk is now known as rwsu | 16:39 | |
| zaro | morning | 16:40 |
| *** mpaolino has quit IRC | 16:41 | |
| *** marcusvrn has quit IRC | 16:42 | |
| *** mpaolino has joined #openstack-infra | 16:42 | |
| openstackgerrit | Anita Kuno proposed openstack-infra/project-config: Change default behaviour of -nv https://review.openstack.org/159916 | 16:42 |
| *** stevemar3 is now known as stevemar | 16:43 | |
| *** stevemar is now known as not-stevemar | 16:43 | |
| *** erlon_away has joined #openstack-infra | 16:44 | |
| openstackgerrit | Ralf Haferkamp proposed openstack-dev/pbr: "packages" might list multiple directories https://review.openstack.org/159918 | 16:44 |
| openstackgerrit | Anita Kuno proposed openstack-infra/project-config: Make devstack-gate stable jobs non-voting https://review.openstack.org/159900 | 16:45 |
| anteaya | morning zaro | 16:46 |
| *** not-stevemar has quit IRC | 16:46 | |
| *** stevemar has joined #openstack-infra | 16:46 | |
| *** sdake__ has quit IRC | 16:47 | |
| openstackgerrit | Anita Kuno proposed openstack-infra/project-config: Change default behaviour of -nv https://review.openstack.org/159916 | 16:48 |
| openstackgerrit | Anita Kuno proposed openstack-infra/project-config: Change default behaviour of -nv https://review.openstack.org/159916 | 16:48 |
| *** sdake_ has joined #openstack-infra | 16:49 | |
| *** carl_baldwin has quit IRC | 16:49 | |
| zaro | jeblair, mordred : would you be able to look at this one? https://review.openstack.org/#/c/155202/ | 16:50 |
| jedimike | are you having problems with Routes-2.1? Looks like the new wheel is broken, fix for it last time was to exclude version 2.0 https://review.openstack.org/#/c/103991/ and we're seeing broken wheel for 2.1 | 16:50 |
| openstackgerrit | Anita Kuno proposed openstack-infra/project-config: Make devstack-gate stable jobs non-voting https://review.openstack.org/159900 | 16:51 |
| *** tonytan4ever has joined #openstack-infra | 16:52 | |
| jeblair | zaro: done | 16:53 |
| openstackgerrit | JJ Asghar proposed openstack-infra/project-config: Updated the Chef repo description https://review.openstack.org/159664 | 16:53 |
| *** ivar-lazzaro has joined #openstack-infra | 16:54 | |
| *** gyee has joined #openstack-infra | 16:55 | |
| *** tonytan4ever has quit IRC | 16:57 | |
| *** mrmartin has quit IRC | 16:57 | |
| *** ivar-lazzaro has quit IRC | 16:59 | |
| *** SumitNaiksatam has quit IRC | 17:02 | |
| *** sabeen1 has joined #openstack-infra | 17:03 | |
| *** mrmartin has joined #openstack-infra | 17:04 | |
| openstackgerrit | Mike Heald proposed openstack/requirements: Exclude Routes 2.1 https://review.openstack.org/159925 | 17:04 |
| *** mpaolino has quit IRC | 17:06 | |
| *** mpaolino has joined #openstack-infra | 17:07 | |
| zaro | jeblair: thanks. change https://review.openstack.org/#/c/155208 'depends-on' the one you just approved but i don't see it in the zuul queue yet. shouldn't it go thru another merge check? | 17:07 |
| *** dtantsur is now known as dtantsur|afk | 17:08 | |
| *** vdo has quit IRC | 17:08 | |
| *** annegentle has quit IRC | 17:09 | |
| jeblair | zaro: enqueuing of reverse dependencies via Depends-On is not automatic; so someone needs to approve that again | 17:10 |
| *** HeOS has quit IRC | 17:12 | |
| *** matrohon has quit IRC | 17:12 | |
| *** unicell1 has quit IRC | 17:13 | |
| jeblair | zaro: (probably after the dependency lands) | 17:13 |
| anteaya | I wonder if recheck would get it moving? | 17:14 |
| *** amotoki has quit IRC | 17:14 | |
| clarkb | recheck won't because the +1 verified doesn't change so gerrit doesn't emit the proper event | 17:16 |
| clarkb | I am going to try testing my image build image caching fix now | 17:17 |
| *** sputnik13 has joined #openstack-infra | 17:17 | |
| *** MaxV has quit IRC | 17:18 | |
| anteaya | clarkb: ah okay | 17:18 |
| clarkb | anteaya: it is a gerrit optimization where if the vote value doesn't change the event for the coment added doesn't include the value | 17:19 |
| clarkb | its a bit annoying since its supposed to be an event system and revoting +1 is a new event ... | 17:19 |
| anteaya | ah okay | 17:19 |
| anteaya | thanks | 17:19 |
| *** nfedotov has quit IRC | 17:19 | |
| clarkb | ok https://review.openstack.org/#/c/159693/ appears to work for at least dib builds | 17:21 |
| *** marcusvrn has joined #openstack-infra | 17:21 | |
| *** _nadya_ has quit IRC | 17:22 | |
| openstackgerrit | Merged openstack-infra/infra-manual: Add clarifications about when to file an ER query https://review.openstack.org/151221 | 17:22 |
| openstackgerrit | Merged openstack-infra/infra-manual: Elaborate on leaving Change-Id alone https://review.openstack.org/153025 | 17:23 |
| clarkb | and my reading of the snapshot builds is that we explicitly run the cache script as the jenkins user during snapshots so it should need the same fix | 17:23 |
| *** reed has quit IRC | 17:24 | |
| *** ZZelle has quit IRC | 17:24 | |
| *** jogo is now known as flashgordon | 17:25 | |
| *** jistr has quit IRC | 17:25 | |
| *** marcusvrn1 has quit IRC | 17:25 | |
| clarkb | mtreinish: "Caching ubuntu_mysql.qcow2 file from http://tarballs.openstack.org/trove/images/ubuntu_mysql.qcow2/ubuntu_mysql.qcow2" from a local dib build using my patch | 17:26 |
| pleia2 | good morning | 17:27 |
| mtreinish | clarkb: cool | 17:27 |
| SpamapS | spock died. :( | 17:27 |
| anteaya | no | 17:28 |
| anteaya | :( | 17:28 |
| anteaya | I heard he was in the hospital | 17:28 |
| *** derekh has quit IRC | 17:29 | |
| SpamapS | dead | 17:29 |
| SpamapS | :( | 17:29 |
| pleia2 | :'( | 17:29 |
| * SpamapS should respect the vulcan culture and not :( | 17:29 | |
| pleia2 | I'm going back to bed! | 17:30 |
| cinerama | he's only 1/2 vulcan | 17:30 |
| pleia2 | (not really) | 17:30 |
| SpamapS | :/ ? | 17:30 |
| cinerama | so some amount of feels are OK :) | 17:30 |
| SpamapS | It's like the scene in Christmas Story where the dogs eat the turkey... | 17:30 |
| SpamapS | "No more Nimoy pictures, no signatures, no convention visits, no time travel to talk to future-old-spock" | 17:31 |
| *** otter768 has joined #openstack-infra | 17:31 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/infra-manual: Document Cross-Repo Dependencies https://review.openstack.org/158855 | 17:32 |
| zaro | clarkb: that optimization can be made optional if that helps. | 17:33 |
| *** dizquierdo has quit IRC | 17:34 | |
| cinerama | now i want to watch the Futurama ST episode (though we were missing De Kelley even then :() | 17:34 |
| *** johnthetubaguy is now known as zz_johnthetubagu | 17:34 | |
| clarkb | zaro: I am thinking it should be, anytime we have an event the entire info for the event should be reflected IMO | 17:34 |
| clarkb | zaro: because these fields aren't necessarily cacheable | 17:34 |
| jeblair | SpamapS: oh no. i am saddened. | 17:35 |
| *** belmoreira has quit IRC | 17:35 | |
| zaro | pleia2: have time to take a look? https://review.openstack.org/#/c/158134/ and https://review.openstack.org/#/c/158387/ | 17:35 |
| *** sdake_ has quit IRC | 17:35 | |
| pleia2 | clarkb: so I could use some insight with a couple procedural things re: new backups server 1) should we useradd them all again like on sysadmin.html#backups or just copy users over from /etc/passwd /etc/shadow, etc? | 17:35 |
| pleia2 | 2) do we just rsync existing data from the old server to the new, or start fresh on backups? | 17:35 |
| cinerama | http://upload.wikimedia.org/wikipedia/commons/4/44/Star_Trek_cast_and_Space_Shuttle_Enterprise.jpg | 17:36 |
| clarkb | pleia2: I would adduser locally simply because the homedir is changing, then rsync/scp the authorized keys over (I think that reduces the number of things that may break given the homedir move) | 17:36 |
| pleia2 | zaro: I'll add them to my queue, but no promises :) | 17:36 |
| clarkb | pleia2: for 2) I think jeblair wants to rsync the existing backup data over | 17:36 |
| pleia2 | clarkb: | 17:37 |
| pleia2 | clarkb: ok, thanks :) | 17:37 |
| *** sputnik13 has quit IRC | 17:37 | |
| zaro | clarkb: hmm, i don't think gerrit community shares that view. but i think they would be open to make it optional, like a gerrit.config option. i'll look into that. | 17:38 |
| clarkb | zaro: I don't understand why not, if this were purely for a db update I could see not doing it, but an event should encompass the entire data of the event | 17:38 |
| clarkb | zaro: esepcially since listeners may not have heard the initial event that set the verified value to +1 | 17:38 |
| *** sputnik13 has joined #openstack-infra | 17:39 | |
| clarkb | but I haven't broght it up with upstream because I am sure you are right that this is intentional and that I am wrong | 17:39 |
| *** e0ne has quit IRC | 17:39 | |
| jeblair | clarkb: didn't you write a patch? | 17:40 |
| openstackgerrit | Andrey Pavlov proposed openstack-infra/project-config: Create our own rally job for ec2-api https://review.openstack.org/159884 | 17:40 |
| clarkb | jeblair: that was a different patch which ended up not being needed | 17:40 |
| *** erw_ is now known as erw | 17:41 | |
| clarkb | jeblair: the patch was to update the timestamp on a comment if a new comment was posted and newer gerrit did that properly (so the upgrade fixed it for us) | 17:41 |
| anteaya | I just read the ny times article and realized the voice in my head was his | 17:42 |
| *** otter768 has quit IRC | 17:43 | |
| jeblair | clarkb: ah right, that was what i was remembering | 17:43 |
| *** mpaolino has quit IRC | 17:43 | |
| jeblair | or mis-remembering | 17:43 |
| jgriffith | mriedem: fixed the sp errors :) | 17:44 |
| *** hdd has joined #openstack-infra | 17:44 | |
| *** skolekonov has quit IRC | 17:44 | |
| clarkb | arosen: can you abandon 157899? will remove it from reviewers review lists | 17:45 |
| *** unicell has joined #openstack-infra | 17:45 | |
| *** _nadya_ has joined #openstack-infra | 17:46 | |
| *** abhishek_cb has joined #openstack-infra | 17:47 | |
| openstackgerrit | Elizabeth K. Joseph proposed openstack-infra/system-config: Update docs for bup user home in /opt/backups https://review.openstack.org/159933 | 17:47 |
| *** salv-orlando has joined #openstack-infra | 17:48 | |
| abhishek_cb | hi all, in my Ci build is not starting while giving any event in gerrit sandbox project, can anyone help me resolving this issue?? | 17:48 |
| *** AJaeger has joined #openstack-infra | 17:49 | |
| *** markvoelker has quit IRC | 17:51 | |
| *** Ryan_Lane has joined #openstack-infra | 17:51 | |
| AJaeger | infra team, I cleaned up our IRC channels with patch https://review.openstack.org/#/c/159585. My ping on some channels got already fixes for three channels and thus I updated the patch. | 17:51 |
| clarkb | AJaeger: awesome | 17:51 |
| AJaeger | I would like this patch and the sorting of the channel list - https://review.openstack.org/#/c/159818/ and https://review.openstack.org/#/c/159838/ merged quickly if you agree with the way forward | 17:52 |
| clarkb | abhishek_cb: are you using zuul? if so check the zuul logs | 17:52 |
| *** teran_ has joined #openstack-infra | 17:52 | |
| AJaeger | Especially the last patch might cause merge conflicts but I would rather have it in soon that rebasing too often - and it should reduce conflicts in the future | 17:52 |
| *** patrickeast has joined #openstack-infra | 17:53 | |
| clarkb | for ianw's f21 dib images to work I need to restart nodepoold to pick up the change that fixed release info if it is an integer in the yaml file | 17:53 |
| clarkb | jeblair: mordred pleia2 I am going to restart nodepoold now to get that done if there is no opposition | 17:53 |
| pleia2 | clarkb: cool, I think some of the builds could use the changes | 17:54 |
| abhishek_cb | clarkb : in zuul logs the zuul is watching them and also no error is coming but it is not triggering any event to german | 17:54 |
| AJaeger | thanks, clarkb! | 17:54 |
| jeblair | clarkb: wfm | 17:55 |
| jeblair | https://review.openstack.org/#/q/project:openstack/governance+status:open+owner:1,n,z | 17:56 |
| jeblair | i proposed 3 governance changes that have infra implications. i don't think they are controversial, just fyi. | 17:56 |
| *** teran has quit IRC | 17:56 | |
| clarkb | restarted, initial log output looks good | 17:57 |
| *** teran_ has quit IRC | 17:57 | |
| jeblair | will add to infra meeting agenda | 17:57 |
| clarkb | so I guess just keep an eye on it but the list of changes made were relatively small and often quite well tested | 17:57 |
| openstackgerrit | Andrey Pavlov proposed openstack-infra/project-config: Create our own rally job for ec2-api https://review.openstack.org/159884 | 17:58 |
| *** funzo has quit IRC | 17:59 | |
| *** mrmartin has quit IRC | 17:59 | |
| jpeeler | can i get a quick gate sanity check here? http://logs.openstack.org/50/158550/4/check/gate-tuskar-python27/6bfea52/console.html check_update.sh fails in the gate, but works locally | 17:59 |
| jpeeler | the sample config file shouldn't need updating, so i'm very confused | 18:00 |
| clarkb | jpeeler: we always test with your change rebased against latest target branch, from the log that can be fetched with `git fetch http://zm01.openstack.org/p/openstack/tuskar refs/zuul/master/Z7d681670fc1842b6b57add73a82abcfe` can you test that ref? | 18:01 |
| clarkb | don't forget to git checkout FETCH_HEAD after fetching it | 18:01 |
| mtreinish | jpeeler: the gate is probably pulling in the latest oslo.db release which isn't updated in tuskar's sample config file | 18:01 |
| clarkb | abhishek_cb: the zuul log indicates that the event is received? if so I would double check your pipeline config to make sure the event matches | 18:02 |
| mtreinish | jpeeler: recreate your tox venv when you run the test locally and you should see it too | 18:02 |
| *** annegentle has joined #openstack-infra | 18:02 | |
| * jpeeler recreates venv | 18:02 | |
| abhishek_cb | clarkb : well is there anything else with the pipeline config I had to do to make it working?? | 18:03 |
| clarkb | abhishek_cb: it depends on what your config is and what events you want to trigger jobs on | 18:04 |
| *** jerryz_ has quit IRC | 18:04 | |
| *** chlong has joined #openstack-infra | 18:04 | |
| *** jaypipes has joined #openstack-infra | 18:05 | |
| *** markvoelker has joined #openstack-infra | 18:05 | |
| abhishek_cb | clarkb: right now it is the fresh CI so basically it will run on comment-added event in sandbox project | 18:06 |
| *** annegentle has quit IRC | 18:07 | |
| *** annegentle has joined #openstack-infra | 18:07 | |
| clarkb | flashgordon: what do you think about adding aiopcpu jobs to neutron experimental so we can debug those failures with potential neutron fixes too? | 18:07 |
| jpeeler | mtreinish: thanks, i didn't even think about the venv being stale | 18:08 |
| *** ssam2 has quit IRC | 18:08 | |
| jpeeler | mtreinish: though now i'm getting and ImportError, presumably because olso config has evolved | 18:08 |
| clarkb | abhishek_cb: does it need to be a specific comment like `recheck`? regardless the other thing I would check is that you have nodes available to run the jobs and that they are registed with gearman | 18:08 |
| mtreinish | jpeeler: np, the new oslo config options is a common problem with running check_uptodate it's why most projects have abandonded using it | 18:08 |
| *** marun has joined #openstack-infra | 18:09 | |
| clarkb | abhishek_cb: connect to gearman (by default on port 4730 of your zuul host) using something like `telnet localhost 4730` on your zuul host and enter `status` to get a registered jobs listing | 18:09 |
| abhishek_cb | clarkb: it is showing the same as mentioned in join fu blog, no issue with that | 18:10 |
| clarkb | abhishek_cb: so you have the job available to run your test against the sandbox repo? | 18:10 |
| openstackgerrit | Merged openstack-infra/project-config: Cleanup accessbot and gerritbot https://review.openstack.org/159585 | 18:10 |
| abhishek_cb | clakb: yes | 18:10 |
| openstackgerrit | Khai Do proposed openstack-infra/project-config: Refactor Gerrit plugin build jobs https://review.openstack.org/157603 | 18:10 |
| openstackgerrit | Khai Do proposed openstack-infra/project-config: Add a job for Gerrit javamelody plugin ver 2.9 https://review.openstack.org/155208 | 18:10 |
| *** chlong has quit IRC | 18:11 | |
| clarkb | abhishek_cb: sounds like zuul doesn't think the generated events match the pipeline then | 18:11 |
| clarkb | I would double check that you are creating events that match your pipelien config | 18:11 |
| abhishek_cb | clarkb : from my side what can i do to resolve this, any suggestions?? | 18:12 |
| clarkb | abhishek_cb: check the event content, do a manual gerrit stream events over ssh to get the info and cross check the included data with your pipeline config | 18:13 |
| abhishek_cb | clarkb : thanks will proceed with the same | 18:13 |
| *** _nadya_ has quit IRC | 18:14 | |
| clarkb | mordred: your input on https://review.openstack.org/#/c/159693/ would be much appreciated | 18:14 |
| *** armax has quit IRC | 18:16 | |
| *** achanda has joined #openstack-infra | 18:16 | |
| openstackgerrit | Merged openstack-infra/project-config: Sort accessbot/channels.yaml https://review.openstack.org/159818 | 18:16 |
| zaro | pelix: just a comment on https://review.openstack.org/#/c/158350/ | 18:16 |
| *** krtaylor has quit IRC | 18:16 | |
| mordred | clarkb: users shoudl exist -that's operating in the chroot after puppet is run | 18:16 |
| *** bdpayne has joined #openstack-infra | 18:17 | |
| mordred | clarkb: so you should be able to chown as needed, if youd' rather do that | 18:17 |
| *** achanda has quit IRC | 18:17 | |
| clarkb | mordred: no its way before puppet runs | 18:17 |
| *** achanda has joined #openstack-infra | 18:17 | |
| clarkb | mordred: this is one of the first things that happens | 18:17 |
| *** carl_baldwin has joined #openstack-infra | 18:18 | |
| *** cody-somerville has joined #openstack-infra | 18:18 | |
| *** dimtruck is now known as zz_dimtruck | 18:18 | |
| mordred | oh - well hrm | 18:18 |
| mordred | clarkb: so - we could move its order, or also I don't hink the sudo call is evil either | 18:19 |
| clarkb | I think it runs early like that beacuse its creating the file that tells dib what to cache for the rest of the build | 18:19 |
| clarkb | so its bootstrapping a major portion of the image build | 18:19 |
| AJaeger | jeblair, anteaya Thanks a lot for the reviews and approvals! | 18:20 |
| clarkb | hrm actually maybe thats only true for a small portion of the extra data | 18:21 |
| AJaeger | clarkb: I'm not insisting on the chown - just wondering whether that would be easier... | 18:21 |
| clarkb | and maybe we can split things up a bit more? | 18:21 |
| openstackgerrit | Jaume Devesa proposed openstack-infra/project-config: Add non-voting tempest gate for Midonet https://review.openstack.org/159867 | 18:21 |
| SpamapS | mordred: So I think you have to do a release of os-client-config so https://review.openstack.org/#/c/159674/ can pass.... | 18:22 |
| SpamapS | mordred: but I also think it might make sense to add a few more unit tests to send a few more lines of code through the py34 shredder first. :) | 18:22 |
| mordred | SpamapS: ++ | 18:23 |
| mordred | on the testing | 18:23 |
| openstackgerrit | Monty Taylor proposed openstack-infra/shade: Add support for keystone projects https://review.openstack.org/159941 | 18:23 |
| mordred | morganfainberg: ^^ | 18:23 |
| mordred | morganfainberg: feedback on that would be most welcome | 18:23 |
| morganfainberg | will do... post coffee ;) | 18:23 |
| mordred | morganfainberg: no rush | 18:24 |
| *** abhishek_cb has quit IRC | 18:24 | |
| morganfainberg | also yay shade! | 18:24 |
| openstackgerrit | Maru Newby proposed openstack-infra/project-config: Limit target projects for the neutron func job https://review.openstack.org/155958 | 18:24 |
| morganfainberg | :) | 18:24 |
| *** e0ne has joined #openstack-infra | 18:24 | |
| morganfainberg | i plan on digging into shade and doing a mass "open bugs for people" and use it as references this weekend | 18:24 |
| mordred | clarkb: honestly, it might be worth diving deep in to how we're doing that with greghaynes | 18:24 |
| morganfainberg | provided i have time. | 18:24 |
| *** e0ne is now known as e0ne_ | 18:24 | |
| openstackgerrit | Merged openstack-infra/project-config: Sort gerritbot/channels.yaml https://review.openstack.org/159838 | 18:25 |
| mordred | clarkb: there is a thing we're doing which is a hack because there wasn't a more dib way to do something | 18:25 |
| clarkb | mordred: it may be but also our images are pretty broken for stable jobs in the mean time | 18:25 |
| mordred | clarkb: sure - no, I mean, for now, we should just land the sudo patch | 18:25 |
| anteaya | AJaeger: thanks for fixing up those files and bringing them up to date | 18:25 |
| mordred | but if we can get greghaynes head wrapped around the dance that's going on there of needed to clone a repo to get values from which we construct a list of repos to clone | 18:26 |
| mordred | all of which we want hooked in to the caching mechanism already in place | 18:26 |
| clarkb | gotcha | 18:26 |
| mordred | clarkb: I have a hunch betterer in the future might be forthcoming :) | 18:26 |
| mordred | clarkb: I'm going to +3 the current patch - k | 18:26 |
| clarkb | thank you | 18:26 |
| mordred | because win now | 18:26 |
| *** ivar-lazzaro has joined #openstack-infra | 18:27 | |
| clarkb | mordred: I question because I am still waiting on ansible and the perfect fix for that :P | 18:27 |
| *** jaypipes has quit IRC | 18:27 | |
| * mordred hides | 18:27 | |
| * clarkb acknowledges that sometimes hacks are hacky but being broken is also :( | 18:28 | |
| *** krtaylor has joined #openstack-infra | 18:28 | |
| *** zz_dimtruck is now known as dimtruck | 18:28 | |
| *** chlong has joined #openstack-infra | 18:28 | |
| * nibalizer nods | 18:28 | |
| *** e0ne_ is now known as e0ne | 18:29 | |
| *** yolanda_ has joined #openstack-infra | 18:29 | |
| *** e0ne has quit IRC | 18:30 | |
| *** jhutton has joined #openstack-infra | 18:31 | |
| openstackgerrit | Ihar Hrachyshka proposed openstack-infra/project-config: Revert "Revert "Temporarily disable swift-dsvm-functional for Icehouse"" https://review.openstack.org/159943 | 18:32 |
| openstackgerrit | Ihar Hrachyshka proposed openstack-infra/project-config: Revert "Revert "Temporarily disable swift-dsvm-functional for Icehouse"" https://review.openstack.org/159943 | 18:33 |
| jhutton | Hi, does anyone know if master tempest still supports stable/icehouse? | 18:33 |
| openstackgerrit | Ihar Hrachyshka proposed openstack-infra/project-config: Revert "Revert "Temporarily disable swift-dsvm-functional for Icehouse"" https://review.openstack.org/159943 | 18:33 |
| jhutton | It seems that master tempest requires tempest-lib, but tempest-lib does not get installed in stable/icehouse devstack | 18:34 |
| clarkb | jhutton: #openstack-qa would know for usre but my understanding is yes master tempest should work against icehouse juno and master (kilo) | 18:34 |
| openstackgerrit | Merged openstack-infra/project-config: Use sudo when manipulating git for image list https://review.openstack.org/159693 | 18:34 |
| clarkb | jhutton: tempest is expected to be run from a virtualenv now using tox | 18:34 |
| *** e0ne has joined #openstack-infra | 18:35 | |
| *** david-lyle_afk is now known as david-lyle | 18:35 | |
| jhutton | yes, i see that being done in master devstack but stable/icehouse doesn't do that yet | 18:35 |
| openstackgerrit | Andrey Pavlov proposed openstack-infra/project-config: Create our own rally job for ec2-api https://review.openstack.org/159884 | 18:35 |
| clarkb | jhutton: I think it does, devstack-gate runs tempest and its using tox for all branches I think | 18:36 |
| clarkb | (devstack does not run tempest) | 18:36 |
| jhutton | clarkb: hmm, but if you do a search for "install_tempest_lib" in stable/icehouse devstack, that function exists but it is not used anywhere | 18:37 |
| *** erlon_away is now known as erlon | 18:37 | |
| clarkb | I don't think it is necessary if running tmpest with tox | 18:38 |
| clarkb | its possible that just wasn't cleaned up properly when the switch to tox virtualenvs was made | 18:38 |
| openstackgerrit | Michael Krotscheck proposed openstack-infra/storyboard-webclient: Tweak for Superuser UI in projects. https://review.openstack.org/159947 | 18:38 |
| openstackgerrit | Michael Krotscheck proposed openstack-infra/storyboard-webclient: User typeahead permits unassign. https://review.openstack.org/159948 | 18:38 |
| openstackgerrit | Elizabeth K. Joseph proposed openstack-infra/puppet-zanata: Initial commit of Zanata-specific sections https://review.openstack.org/147947 | 18:38 |
| *** chlong has quit IRC | 18:38 | |
| jhutton | clarkb: i'm still using "testr run" so maybe that's why it's not working for me | 18:40 |
| *** e0ne is now known as e0ne_ | 18:40 | |
| clarkb | jeblair: https://review.openstack.org/#/c/157592/ are those failures familiar to you? I don't think they are related to my change but I am looking at it | 18:40 |
| *** e0ne_ is now known as e0ne | 18:41 | |
| clarkb | jeblair: looks like a race around aborting some jobs where it is possible for them to get through successfully first | 18:41 |
| *** tonytan4ever has joined #openstack-infra | 18:41 | |
| clarkb | thanks everyone for reviews on 156788, I will babysit that when I get a moment | 18:42 |
| jeblair | clarkb: not familiar, sorry | 18:42 |
| *** teran has joined #openstack-infra | 18:43 | |
| clarkb | jeblair: are you comfortable with me rechecking that change or should I hold off and debug the race condition a bit more? | 18:43 |
| openstackgerrit | Anita Kuno proposed openstack-infra/project-config: Make devstack-gate stable jobs non-voting https://review.openstack.org/159900 | 18:43 |
| jeblair | clarkb: i think is okay to recheck; someone should probably try to fix it though. i'd be happy to, but won't be able to for several hours | 18:44 |
| clarkb | ya I need to go do morning things then prep for ride downtown, maybe will leave it as is for the moment and use bus time for mental code path tracing | 18:45 |
| openstackgerrit | Ihar Hrachyshka proposed openstack-infra/project-config: Revert "Revert "Temporarily disable swift-dsvm-functional for Icehouse"" https://review.openstack.org/159943 | 18:45 |
| *** bswartz has joined #openstack-infra | 18:46 | |
| *** nelsnels_ has joined #openstack-infra | 18:48 | |
| greghaynes | clarkb: mordred hrm? re: users being created | 18:49 |
| *** teran_ has joined #openstack-infra | 18:51 | |
| *** koolhead17 has joined #openstack-infra | 18:51 | |
| *** hdd has quit IRC | 18:52 | |
| greghaynes | Maybe I can get clarkb to give me the full context dump at lunch :) | 18:53 |
| *** teran has quit IRC | 18:53 | |
| *** yamahata has joined #openstack-infra | 18:53 | |
| *** reed has joined #openstack-infra | 18:54 | |
| mordred | greghaynes: we're abusing dib for some things | 18:54 |
| *** jamesmcarthur has quit IRC | 18:54 | |
| mordred | greghaynes: and itmight be useful to have a smart person have an alternate theory of doing the right thing | 18:54 |
| *** cdent has quit IRC | 18:55 | |
| greghaynes | dib is great at being abused | 18:55 |
| mordred | greghaynes: basically, we have a copy of the source repos code that we use to clone a source repo before source repos runs, we then use the contents of that source repo to generate a source-repos file to be input to the real source repos command | 18:55 |
| mordred | this is, clearly, wrong | 18:56 |
| greghaynes | hah, ok | 18:56 |
| *** tosky has quit IRC | 18:56 | |
| mordred | jeblair: actually - any reason we have to read projects.yaml and don't just do gerrit ls-projects? | 18:57 |
| arosen | clarkb abandon'ed | 18:57 |
| *** markvoelker has quit IRC | 18:58 | |
| *** funzo has joined #openstack-infra | 18:59 | |
| *** AJaeger has quit IRC | 19:00 | |
| *** annegentle has quit IRC | 19:01 | |
| *** AJaeger has joined #openstack-infra | 19:01 | |
| *** markvoelker has joined #openstack-infra | 19:01 | |
| morganfainberg | mordred, commented on shade patch. the only real issue i see is you're goign to run into cases where you wont get the project you expect if using V3 and not adding in domain support. | 19:01 |
| *** AJaeger has quit IRC | 19:01 | |
| openstackgerrit | Merged openstack-infra/project-config: Add a job for Gerrit javamelody plugin ver 2.9 https://review.openstack.org/155208 | 19:03 |
| clarkb | yes I will fill greghaynes in at lunch | 19:04 |
| clarkb | arosen: thank you | 19:04 |
| clarkb | mordred: so I have been thinking about the ansible puppet stuff and I think there are two major steps in it. The first is getting ansible to write supplementary fact files that augment or extend the master's hiera data so that we can fix this race with git checkouts. The second is full on write hiera files (and ditch the facts) for local puppet apply | 19:05 |
| mordred | morganfainberg: so - do I need to pass in a domain to the project search stuff? or if I connected with a domain scoped auth will it be ok | 19:05 |
| morganfainberg | mordred, i am fairly certain you need to pass domain to project search, so keystone knows to filter on that | 19:06 |
| mordred | clarkb: did you look at my patch to ansible-puppet? | 19:06 |
| mordred | morganfainberg: ok. | 19:06 |
| clarkb | mordred: no, I keep forgetting that is a repo now | 19:06 |
| * clarkb adds it to the watch list | 19:06 | |
| mordred | clarkb: I need to add an example usage to it from jeblair's feedback, I'll get that done today | 19:07 |
| morganfainberg | i think we have a bug in ksc. | 19:07 |
| clarkb | mordred: I was hoping you wouldn't do it in the module... | 19:07 |
| * morganfainberg goes and looks at something | 19:08 | |
| clarkb | mordred: because in theory ansible can dump json places without any effort | 19:08 |
| mordred | nope | 19:08 |
| *** melwitt has joined #openstack-infra | 19:08 | |
| mordred | total fabrication | 19:08 |
| morganfainberg | nope no bug. nvm. | 19:08 |
| jeblair | mordred: http://git.openstack.org/cgit/openstack-infra/project-config/tree/nodepool/scripts/cache_git_repos.py#n70 | 19:08 |
| mordred | clarkb: that was my first thought | 19:08 |
| jeblair | mordred: is that what you're talking about? | 19:08 |
| mordred | jeblair: yup. so we should to that TODO :) | 19:09 |
| *** nelsnels_ has quit IRC | 19:09 | |
| jeblair | mordred: yep. rest api did not exist at the time. ls-projects requires an ssh key. but rest api does not, so it should be fine. | 19:09 |
| *** mriedem has quit IRC | 19:09 | |
| mordred | jeblair: that might actually make the chicken-and-egg weirdness we do there much simpler | 19:09 |
| mordred | well, in the dib part | 19:09 |
| zxiiro | anyone available / mind to review this patch? https://review.openstack.org/#/c/155865/ | 19:09 |
| jeblair | mordred: note however, that is still at least just doing http -- it's not doing a git clone | 19:09 |
| jeblair | mordred: that's basically a curl on the projects.yaml url | 19:10 |
| clarkb | jeblair: mordred that isn't related to the chicken and eggness fwiw | 19:10 |
| *** nelsnelson has joined #openstack-infra | 19:10 | |
| clarkb | the chicke and egg is specific to devstack | 19:10 |
| mordred | clarkb: RIIIIGH | 19:10 |
| *** BharatK has quit IRC | 19:10 | |
| mordred | thank yo | 19:10 |
| jeblair | we're talking about devstack? | 19:10 |
| mordred | I forgot what the chicken and egg was - brain correcet | 19:10 |
| clarkb | we need devstack checked out in order to get the list of images to cache | 19:10 |
| mordred | yup. I remember now | 19:10 |
| clarkb | and we need it early enough for the dib caching to do the work for us | 19:10 |
| jeblair | clarkb: yep; i saw talk about listing projects for caching which is what i was responding to | 19:11 |
| mordred | jeblair: I asked the wrong question | 19:11 |
| *** radez has quit IRC | 19:11 | |
| jeblair | ok | 19:11 |
| *** mriedem has joined #openstack-infra | 19:12 | |
| clarkb | mordred: ok that ansible-puppet change lines up mostly with that I expected it to look like. Was hoping asnible made it easier though but can live with that | 19:13 |
| mordred | clarkb: in any case - you seem to have this aversion to library files in ansible and an interest in writing logic in the yaml files - and I'd like to connect on that | 19:13 |
| mordred | cool | 19:13 |
| clarkb | mordred: yes for similar reasons to why we don't write puppet providers for everything | 19:13 |
| mordred | well, we dont' do that because we hate ruby | 19:13 |
| mordred | it's probably the right thing to do more of the time than we do it | 19:13 |
| clarkb | mordred: it encodes a bunch of generically useful info in a non generic way and means any time you want to do it someplace else its a complete rewrite | 19:13 |
| nibalizer | kindof | 19:13 |
| mordred | but also - puppet's dsl is a logic language | 19:13 |
| nibalizer | its a real pain | 19:13 |
| mordred | ansible's yaml files are to encode a sequence of commands | 19:13 |
| clarkb | mordred: right and commands are rich enough to do ^ | 19:14 |
| clarkb | mordred: supposedly ansible knows how to take an internal data structure and just write a .json somewhere with it | 19:14 |
| clarkb | so its not even exec bash but an intneral ansible command | 19:14 |
| mordred | clarkb: I do not belive that to be the case - but I will certainly keep looking for reference of it | 19:14 |
| clarkb | mordred: but my aversion is config managment agnostic. I think people like to layer on a bunch of their own logic which isn't necessary /me looks at that JJB puppet module | 19:15 |
| greghaynes | this is what we did in tripleo-ansible also | 19:15 |
| mordred | clarkb: totally - I think that just seems to my like what you're suggesting | 19:15 |
| greghaynes | just write out a json blob for os-collect-config to consume | 19:15 |
| *** stevemar has quit IRC | 19:15 | |
| greghaynes | that was a custom module IIRC | 19:15 |
| clarkb | mordred: did you know that a group of people wrote a puppet module that incoporated JJB into the puppet DSL rather than just exec puppet like we do? | 19:15 |
| mordred | greghaynes: link? | 19:15 |
| mordred | clarkb: right. | 19:15 |
| mordred | but that's exactly my point | 19:15 |
| clarkb | and this feels very similar to that | 19:15 |
| mordred | I tink putting logic into the yaml file is the equiv of that | 19:15 |
| mordred | this is "please just do this thing kthxbai" | 19:16 |
| clarkb | we can just run puppet and check a few things via ansible primitives, or we can layer on unneeded abstraction | 19:16 |
| mordred | right - but we can't - because the things we have to check are too complex to do sanely there | 19:16 |
| mordred | like "are these strings contained in stdout" | 19:16 |
| clarkb | mordred: yes puppet certainly doesn't make it easy | 19:16 |
| greghaynes | mordred: http://git.openstack.org/cgit/stackforge/tripleo-ansible/tree/playbooks/refresh_config.yml | 19:16 |
| mordred | there is a logic threshold where python is a nicer language than yaml :) | 19:16 |
| clarkb | which is unfortunate because it should be easy :/ | 19:17 |
| *** stevemar has joined #openstack-infra | 19:17 | |
| mordred | greghaynes: thanks | 19:17 |
| greghaynes | mhttp://git.openstack.org/cgit/stackforge/tripleo-ansible/tree/playbooks/templates/host_metadata.json.j2 | 19:17 |
| greghaynes | that where the magic happens | 19:17 |
| mordred | greghaynes: yes. that was what my first stab looked like | 19:17 |
| clarkb | anyways going back to what I was original trying to get to, I think an important thing that hasn't been explained/solved/fixed is how do we manually run puppet without ansible overwriting our stuff | 19:17 |
| *** jamesmcarthur has joined #openstack-infra | 19:18 | |
| clarkb | I think it involves an ansible lock on a node, maybe by mangling ansible inventory or killing ssh access or similar | 19:18 |
| *** tmcpeak has joined #openstack-infra | 19:18 | |
| *** stevemar has quit IRC | 19:18 | |
| mordred | clarkb: aha! yes, thank you - there is a bug in that code | 19:18 |
| *** stevemar has joined #openstack-infra | 19:18 | |
| *** tmcpeak has quit IRC | 19:18 | |
| mordred | it goes too far before discovering a puppet disable lock | 19:18 |
| clarkb | mordred: right because even if I disable puppet ansible pwns you | 19:18 |
| clarkb | yup | 19:18 |
| mordred | yup | 19:18 |
| *** jamesmcarthur has quit IRC | 19:18 | |
| mordred | k. I'll fix that - thanks! | 19:19 |
| anteaya | jeblair: so this approach fails: https://review.openstack.org/#/c/159900/6/jenkins/jobs/projects.yaml since tempest can't find the -icechouse and -juno jobs it needs | 19:19 |
| anteaya | refining the devstack-jobs affects both projects | 19:19 |
| clarkb | mordred: fwiw I am ok with using the script, its there, it mostly works, I just think it makes using ansible as a tool harder which is where I am grumping | 19:19 |
| jeblair | anteaya: yeah, you'll probably need to add new ones | 19:19 |
| clarkb | because we have this non standard abstraction for a specific thing that isn't part of ansible proper | 19:19 |
| anteaya | copy them and have a designator -icehouse and -icehouse-nv? | 19:20 |
| *** mrmartin has joined #openstack-infra | 19:20 | |
| jeblair | yeah | 19:20 |
| anteaya | kk | 19:20 |
| mordred | clarkb: well, honestly, I'd land it upstream into ansible, actually | 19:21 |
| clarkb | mordred: also I did confirm that /etc/facter/facts.d facts are overridden by env facts, so possible that you only need to deal with file facts in that role | 19:21 |
| clarkb | mordred: then humans can use env facts when running puppet manually | 19:21 |
| mordred | clarkb: cool, I'll just do that | 19:21 |
| mordred | clarkb: landing functionality like this library into ansible upstream is highly normal activity - it's mildly strange in this case specifically because it's a module for running puppet | 19:22 |
| mordred | I have not talked to the new project lead | 19:22 |
| mordred | but last time I talked to michael, he was a little averse to landing it upstream because he didn't want to convey the message that ansible needed another config management system to do config management | 19:23 |
| mordred | but he's not running things anymore | 19:23 |
| mordred | so i'll bring it u | 19:23 |
| mordred | up | 19:23 |
| *** HeOS has joined #openstack-infra | 19:23 | |
| openstackgerrit | Anita Kuno proposed openstack-infra/project-config: Make devstack-gate stable jobs non-voting https://review.openstack.org/159900 | 19:23 |
| *** openstackgerrit has quit IRC | 19:24 | |
| mordred | in general, I agree - things should go into the ansible modules library and just be part of ansible | 19:24 |
| *** openstackgerrit has joined #openstack-infra | 19:24 | |
| SpamapS | mordred: I think Michael was a little off on that one. The community by and large likes Ansible because it orchestrates well. | 19:24 |
| clarkb | mordred: I noted the two things about locking and env facts above on the review | 19:24 |
| mordred | SpamapS: I agree | 19:24 |
| mordred | clarkb: sweet. thanks! | 19:24 |
| SpamapS | ok, on the road for a while.. | 19:25 |
| clarkb | mordred: but I am excited to get this in | 19:25 |
| * SpamapS goes into the quiet place | 19:25 | |
| clarkb | mordred: https://review.openstack.org/#/c/154304/ should be abandoned right? | 19:25 |
| anteaya | SpamapS: travel safely | 19:25 |
| clarkb | mordred: and https://review.openstack.org/#/c/154300/2/manifests/init.pp will need to be updated to use whatever fact we settle on | 19:25 |
| *** vilobhmm has quit IRC | 19:25 | |
| *** jamesmcarthur has joined #openstack-infra | 19:26 | |
| mordred | anteaya: yes and yes | 19:26 |
| *** vilobhmm has joined #openstack-infra | 19:26 | |
| anteaya | what did I ask? | 19:26 |
| mordred | but first - I will now eat food | 19:26 |
| *** ihrachyshka has quit IRC | 19:26 | |
| * anteaya forgets | 19:26 | |
| mordred | anteaya: gah | 19:26 |
| anteaya | enjoy food | 19:26 |
| mordred | clarkb: yes and yes | 19:26 |
| mordred | anteaya: I mistook you and clark | 19:26 |
| anteaya | I can understand how that could happen | 19:26 |
| reed | sdague, when you say "We already purge old stuff that's unmergable" is that a manual process or automatic? | 19:26 |
| mtreinish | reed: there's a script to do it which is manually run | 19:27 |
| clarkb | mordred: thanks, I abandoned that change and can update the second whenever we get far enough to have a fact name | 19:27 |
| reed | mtreinish, what's the criteria that trigger the run? | 19:28 |
| mtreinish | reed: http://git.openstack.org/cgit/openstack/nova/tree/tools/abandon_old_reviews.sh | 19:28 |
| *** marcusvrn1 has joined #openstack-infra | 19:28 | |
| *** jamesmcarthur has quit IRC | 19:28 | |
| clarkb | (it should be noted I think its cool that people like JJB so much they shoved it into the puppet DSL, I just think its the wrong way to go about solving that problem) | 19:28 |
| reed | mtreinish, is that used only in nova or other projects too? | 19:28 |
| mtreinish | reed: it's run periodically by someone who has +2 on the repo | 19:28 |
| mordred | clarkb: so ... follow up question ... | 19:29 |
| mtreinish | reed: I've run it on tempest and I think neutron uses it too | 19:29 |
| reed | ok, thanks mtreinish | 19:29 |
| mordred | clarkb: do we need both hiera and facter output support actually? | 19:29 |
| *** jamesmcarthur has joined #openstack-infra | 19:29 | |
| clarkb | mordred: yes I think so. For the transition where we have a master with hiera data doing local hiera too is potentially painful | 19:29 |
| *** vilobhmm has quit IRC | 19:29 | |
| mordred | clarkb: I meant the other way | 19:29 |
| clarkb | mordred: so facts can be the supplementary data, then when we switch to puppet apply we can kll off facts | 19:29 |
| clarkb | though having fact support in general is probably useful | 19:30 |
| mordred | what value does outputting hiera give us when we can just output facts | 19:30 |
| clarkb | mordred: puppet variable resolution is hiera smart so you can scope things and do other fun stuff | 19:30 |
| mordred | oh - right | 19:30 |
| clarkb | whereas facts are purely top scope | 19:30 |
| mordred | yah. kk | 19:30 |
| mordred | just making sure I wasnt' doing two things for "completeness" and not a reason | 19:30 |
| openstackgerrit | Davanum Srinivas (dims) proposed openstack-infra/devstack-gate: WIP - Do not merge - Trying a different locale https://review.openstack.org/159960 | 19:30 |
| clarkb | mordred: nope I think the general shape of what you have is good | 19:30 |
| *** marcusvrn has quit IRC | 19:31 | |
| *** vilobhmm has joined #openstack-infra | 19:31 | |
| mordred | cool | 19:31 |
| *** ildikov has joined #openstack-infra | 19:31 | |
| *** e0ne has quit IRC | 19:32 | |
| *** yamamoto has joined #openstack-infra | 19:33 | |
| *** radez has joined #openstack-infra | 19:35 | |
| *** cnesa1 has left #openstack-infra | 19:36 | |
| *** e0ne has joined #openstack-infra | 19:36 | |
| *** fifieldt_ has joined #openstack-infra | 19:37 | |
| *** yamamoto has quit IRC | 19:37 | |
| *** gyee has quit IRC | 19:38 | |
| anteaya | yay 159900 passed check tests | 19:39 |
| *** sdake_ has joined #openstack-infra | 19:39 | |
| *** ddieterly has quit IRC | 19:39 | |
| *** ddieterly has joined #openstack-infra | 19:40 | |
| *** fifieldt has quit IRC | 19:40 | |
| *** ddieterly has quit IRC | 19:40 | |
| *** annegentle has joined #openstack-infra | 19:40 | |
| *** ddieterly has joined #openstack-infra | 19:40 | |
| *** hdd has joined #openstack-infra | 19:41 | |
| sigmavirus24 | So https://review.openstack.org/#/c/159931/ was marked as unstable because scp'ing the logs failed. It isn't very clear to me why that failed though. Is there a way I can help debug this/fix it? | 19:43 |
| *** otter768 has joined #openstack-infra | 19:43 | |
| *** doude has quit IRC | 19:44 | |
| *** doude has joined #openstack-infra | 19:47 | |
| *** otter768 has quit IRC | 19:48 | |
| anteaya | well if logs fail to upload it is usually one of two things | 19:49 |
| anteaya | one a host issue with the connection | 19:50 |
| anteaya | two a problem with the log server, usually a memory issue, as in no space | 19:50 |
| anteaya | let's check the log server first | 19:50 |
| *** barra204 has joined #openstack-infra | 19:50 | |
| *** sarob has joined #openstack-infra | 19:50 | |
| *** jerryz has joined #openstack-infra | 19:50 | |
| anteaya | http://cacti.openstack.org/cacti/graph_view.php?action=tree&tree_id=1&leaf_id=7&select_first=true | 19:50 |
| anteaya | static.openstack.org is our log server name | 19:51 |
| anteaya | http://cacti.openstack.org/cacti/graph_view.php?action=tree&tree_id=1&leaf_id=13 | 19:51 |
| anteaya | memory looks good, there is lots of space | 19:52 |
| *** jamesmcarthur has quit IRC | 19:52 | |
| anteaya | cpu, it was into io wait for about 12 hours but that has passed | 19:52 |
| *** rm_work|away is now known as rm_work | 19:52 | |
| *** achanda has quit IRC | 19:53 | |
| anteaya | the test might have run during the period of time the cpu was in io wait | 19:54 |
| anteaya | sigmavirus24: try doing a recheck | 19:55 |
| *** yolanda_ has quit IRC | 19:55 | |
| anteaya | not sure what was happening on the log server for that 12 hour period | 19:55 |
| *** mikeit has joined #openstack-infra | 19:55 | |
| sigmavirus24 | okay | 19:55 |
| sigmavirus24 | I'll try a recheck | 19:55 |
| anteaya | sigmavirus24: thanks for asking | 19:55 |
| sigmavirus24 | Thanks anteaya | 19:55 |
| anteaya | np | 19:55 |
| sigmavirus24 | TIL about cacti.openstack.org | 19:55 |
| anteaya | it is handy | 19:56 |
| *** barra204 is now known as shakamunyi | 19:56 | |
| anteaya | I always get to it from: http://ci.openstack.org/cacti.html | 19:56 |
| *** yolanda_ has joined #openstack-infra | 19:56 | |
| anteaya | as the url needs a .php ending and I always forget the syntax | 19:56 |
| anteaya | hopefully someone with root can take a look at the server logs to see what was going on | 19:57 |
| *** marun has quit IRC | 19:57 | |
| *** jerryz has quit IRC | 19:57 | |
| sigmavirus24 | anteaya: sounds like it's happening on every check | 20:00 |
| sigmavirus24 | We just added a publisher macro to project-config last night it seems | 20:00 |
| * sigmavirus24 didn't realize it was merged already | 20:01 | |
| anteaya | did we | 20:02 |
| *** marun has joined #openstack-infra | 20:02 | |
| *** ChuckC has quit IRC | 20:02 | |
| *** ZZelle has joined #openstack-infra | 20:02 | |
| clarkb | they did, could be a perms problem | 20:02 |
| anteaya | ah | 20:02 |
| clarkb | I double checked about the path because /openstack was interesting :) | 20:03 |
| anteaya | sigmavirus24: on every check on everything or every check on that repo? | 20:03 |
| anteaya | clarkb: do expand | 20:03 |
| *** ChuckC has joined #openstack-infra | 20:03 | |
| clarkb | it just looks like a relative path but I am told they use /openstack rooted on / | 20:03 |
| sigmavirus24 | clarkb: yeah the /openstack/log path is correct | 20:04 |
| anteaya | sigmavirus24: it looks to be like every recent jenkins run on that repo | 20:04 |
| sigmavirus24 | clarkb: so that's where we bind-mount back the service's log directories from the containers | 20:04 |
| sigmavirus24 | anteaya: correct (/me was about to confirm in our channel) | 20:04 |
| sigmavirus24 | Let me check the permissions on this AIO I have running | 20:05 |
| *** EntropyWorks has left #openstack-infra | 20:05 | |
| anteaya | which still doesn't explain what was happening on the log server for that 12 hour period of time | 20:05 |
| clarkb | anteaya there are semi established rules that tell you where logs go /var/log/openstack for example not /openstack/log | 20:05 |
| anteaya | ah | 20:05 |
| clarkb | ya perms would be next thing to check | 20:05 |
| anteaya | are these rules anywhere I can look up | 20:05 |
| anteaya | or you just know them? | 20:05 |
| clarkb | http://en.m.wikipedia.org/wiki/Filesystem_Hierarchy_Standard is a reasonable intro | 20:06 |
| *** yolanda_ has quit IRC | 20:06 | |
| anteaya | ah so rules not specific to us | 20:07 |
| anteaya | even better | 20:07 |
| sigmavirus24 | clarkb: yeah I'm wondering if there were issues creating the bind-mounts back into /var/log/etc. | 20:07 |
| anteaya | and thanks | 20:07 |
| *** gordc has quit IRC | 20:07 | |
| clarkb | there are no hard rules though and /openstack/log will work. it will just make ops people cranky when they cany find the logs | 20:07 |
| *** vilobhmm has quit IRC | 20:07 | |
| clarkb | *cant | 20:07 |
| *** annegentle has quit IRC | 20:07 | |
| *** vilobhmm has joined #openstack-infra | 20:07 | |
| *** markvoelker has quit IRC | 20:08 | |
| *** vilobhmm has quit IRC | 20:08 | |
| *** annegentle has joined #openstack-infra | 20:08 | |
| *** vilobhmm has joined #openstack-infra | 20:08 | |
| anteaya | right | 20:09 |
| anteaya | thanks clarkb I'm an idiot but I like learning stuff | 20:11 |
| clarkb | also worth noting /opt is supposed to be for optional installs which is why we use a lot of it | 20:11 |
| *** markvoelker_ has joined #openstack-infra | 20:12 | |
| clarkb | but devstack puts its logs under its install dir so we have /opt/stack/new/devstack/logs | 20:12 |
| clarkb | which is no better :) | 20:12 |
| *** e0ne is now known as e0ne_ | 20:13 | |
| anteaya | ha ha ha | 20:15 |
| openstackgerrit | Davanum Srinivas (dims) proposed openstack-infra/devstack-gate: WIP - Do not merge - Trying a different locale https://review.openstack.org/159960 | 20:16 |
| *** markvoelker_ has quit IRC | 20:16 | |
| anteaya | clarkb: what do you think was happening on the log server for that 12 hour period of time that cacti shows it was in memory buffer and io wait? | 20:16 |
| clarkb | probably doing its compression run | 20:17 |
| anteaya | ah | 20:17 |
| anteaya | so wouldn't cause a failure to upload files | 20:17 |
| anteaya | how often does it do it's compression run? | 20:18 |
| *** hdd has quit IRC | 20:18 | |
| pleia2 | progress on new backups server: mounted cinder volume and added to fstab, all back users set up and I'm in the process of rsyncing backup data (groups done, etherpad in progress) | 20:19 |
| clarkb | every 4 hours but with alock so if it takes lng it wont start another | 20:19 |
| pleia2 | s/back users/backup users | 20:19 |
| anteaya | pleia2: nice work | 20:19 |
| clarkb | pleia2 woot! | 20:19 |
| anteaya | clarkb: interesting, thanks | 20:19 |
| clarkb | I'm on my way downtown so greghaynes can help me fix our dib | 20:20 |
| pleia2 | woo | 20:20 |
| clarkb | also lunch | 20:21 |
| pelix | zaro: I'll have to look through the reviews over the weekend, just too much to get sorted and I'm already 3 hours late finishing | 20:21 |
| *** achanda has joined #openstack-infra | 20:23 | |
| anteaya | sigmavirus24: so for another time, perhaps a better commit message title? https://review.openstack.org/#/c/159578/ | 20:24 |
| anteaya | sigmavirus24: this is what I have to look through: https://review.openstack.org/#/q/project:openstack-infra/project-config+status:merged,n,z | 20:24 |
| openstackgerrit | Ian Wienand proposed openstack-infra/system-config: Add Fedora 21 image build to nodepool https://review.openstack.org/140911 | 20:24 |
| sigmavirus24 | anteaya: I'll pass that on to rackertom | 20:24 |
| anteaya | thank you | 20:25 |
| anteaya | :) | 20:25 |
| *** jkraj has quit IRC | 20:25 | |
| rackertom | 👍 | 20:25 |
| sigmavirus24 | So looking at the permissions, these files should all be readable by any user on the node | 20:26 |
| *** e0ne_ is now known as e0ne | 20:26 | |
| *** mikeit has quit IRC | 20:29 | |
| *** achanda has quit IRC | 20:30 | |
| *** e0ne has quit IRC | 20:31 | |
| *** ChuckC has quit IRC | 20:31 | |
| *** ChuckC has joined #openstack-infra | 20:32 | |
| *** ddieterly has quit IRC | 20:32 | |
| *** ddieterly has joined #openstack-infra | 20:33 | |
| sigmavirus24 | anteaya: it wouldn't make any sense for the hierarchy to be too deep for that publisher, would it? | 20:33 |
| sigmavirus24 | I mean, would it be plausible that creating openstack/log/aio1_utility-etc/logfile1.log inside of logs/ would be problematic? | 20:34 |
| sigmavirus24 | The directories should be accessible by everyone and their contents readable by everyone (from what I can tell with our AIO) so this seems like a really bizarre problem to be having | 20:35 |
| *** e0ne has joined #openstack-infra | 20:35 | |
| anteaya | sigmavirus24: at this point I will redirect you to clarkb or pleia2 | 20:36 |
| anteaya | as they would be in a better position to intelligently answer this question | 20:36 |
| *** ddieterly has quit IRC | 20:36 | |
| anteaya | I too look forward to the response | 20:37 |
| sigmavirus24 | anteaya: I'm really just grasping for straws here | 20:37 |
| anteaya | all I can do is offer emotional support | 20:38 |
| * anteaya offers emotional support | 20:38 | |
| anteaya | as I don't know either | 20:38 |
| sigmavirus24 | heh | 20:38 |
| sigmavirus24 | thanks anteaya | 20:38 |
| *** markvoelker has joined #openstack-infra | 20:39 | |
| anteaya | best I have to offer atm | 20:39 |
| pleia2 | unfortunately I'm not familiar enough with this yet to give a helpful answer, sorry | 20:39 |
| sigmavirus24 | I'm going to try something else to see if I can reproduce this | 20:40 |
| anteaya | good idea | 20:42 |
| jhesketh | Morning | 20:42 |
| *** ddieterly has joined #openstack-infra | 20:43 | |
| *** e0ne has quit IRC | 20:44 | |
| openstackgerrit | Matt Riedemann proposed openstack-infra/elastic-recheck: Add query for neutron DBDeadlock bug 1426543 https://review.openstack.org/159993 | 20:44 |
| anteaya | morning jhesketh | 20:44 |
| openstack | bug 1426543 in OpenStack Compute (nova) "Spike in DBDeadlock errors in update_floatingip_statuses since 2/27" [Undecided,New] https://launchpad.net/bugs/1426543 | 20:44 |
| anteaya | you know things about logging | 20:44 |
| anteaya | once you get settled, sigmavirus24 has some logging concerns | 20:44 |
| anteaya | in that the new macro on their repo prevents it | 20:44 |
| anteaya | https://review.openstack.org/#/c/159578/ | 20:44 |
| * jhesketh has limited time before his Saturday breakfast and Linux Australia council meeting | 20:44 | |
| anteaya | ah | 20:44 |
| anteaya | saturday for you | 20:45 |
| anteaya | happy saturday | 20:45 |
| jhesketh | but happy to look | 20:45 |
| *** mikeit has joined #openstack-infra | 20:45 | |
| sigmavirus24 | Silly other side of the world | 20:45 |
| jhesketh | silly people in the past ;-) | 20:45 |
| openstackgerrit | Sean Roberts proposed openstack-infra/system-config: patch to create test-certification-standards ilist https://review.openstack.org/159995 | 20:45 |
| anteaya | other side of the world is awesome | 20:45 |
| anteaya | I love it | 20:45 |
| *** markvoelker has quit IRC | 20:45 | |
| jhesketh | anteaya, sigmavirus24: what do you mean by 'prevents it'? | 20:46 |
| sigmavirus24 | jhesketh: let me grab a console log for you | 20:46 |
| anteaya | https://review.openstack.org/#/c/159931/ | 20:46 |
| anteaya | the job marked unstable | 20:46 |
| anteaya | it failed during scp | 20:46 |
| sigmavirus24 | anteaya: is way too fast for me | 20:46 |
| anteaya | you should see when fungi is here | 20:46 |
| anteaya | can'at see him for dust | 20:47 |
| anteaya | all recent jenkins runs on that repo fail with unstable: https://review.openstack.org/#/q/project:stackforge/os-ansible-deployment,n,z | 20:48 |
| *** gordc has joined #openstack-infra | 20:48 | |
| *** MaxV has joined #openstack-infra | 20:48 | |
| sigmavirus24 | oh wow | 20:48 |
| sigmavirus24 | I need to read tracebacks closer | 20:49 |
| sigmavirus24 | Caused by: java.io.IOException: Expecting Ant GLOB pattern, but saw '/openstack/log/**/*.log'. See http://ant.apache.org/manual/Types/fileset.html for syntax | 20:49 |
| *** e0ne has joined #openstack-infra | 20:49 | |
| sigmavirus24 | I thought I had seen **/*.log earlier in the macros file | 20:49 |
| sigmavirus24 | seems **/*.log isn't allowed? | 20:49 |
| anteaya | well tehre is the answer to your question | 20:49 |
| anteaya | so depth matters | 20:49 |
| flashgordon | clarkb: ++ to adding aiopcpu to neutron experimental | 20:50 |
| *** mikeit has quit IRC | 20:50 | |
| *** sarob has quit IRC | 20:50 | |
| sigmavirus24 | anteaya: so https://github.com/openstack-infra/project-config/blob/master/jenkins/jobs/macros.yaml#L529 uses **, which means maybe it just doesn't expect the /*.log | 20:51 |
| sigmavirus24 | although https://github.com/openstack-infra/project-config/blob/master/jenkins/jobs/macros.yaml#L569 works too | 20:51 |
| * sigmavirus24 goes to research ANT's grammar | 20:52 | |
| jhesketh | sigmavirus24: I would have thought that would be okay | 20:52 |
| jhesketh | ie, that glob path look sfine to me | 20:52 |
| sigmavirus24 | jhesketh: same here | 20:52 |
| sigmavirus24 | jhesketh: for normal shells it would be | 20:52 |
| *** markvoelker has joined #openstack-infra | 20:52 | |
| sigmavirus24 | but perhaps ant doesn't care for it | 20:52 |
| * sigmavirus24 is reading | 20:52 | |
| *** e0ne is now known as e0ne_ | 20:53 | |
| jhesketh | it seems to be the only glob that doesn't begin with '**' out of all our jobs | 20:53 |
| jhesketh | maybe it's a permissions thing? | 20:53 |
| sigmavirus24 | jhesketh: looked at permissions already | 20:54 |
| sigmavirus24 | I haven't tried to scp with a non-root user yet though | 20:54 |
| sigmavirus24 | the others don't keep their hierarchy though | 20:54 |
| jhesketh | sigmavirus24: ah, maybe it doesn't know how to glob with hierarchy | 20:55 |
| sigmavirus24 | I think it does | 20:55 |
| sigmavirus24 | because the docs-draft jobs do | 20:55 |
| sigmavirus24 | but they only do /foo/bar/bogus/** | 20:55 |
| sigmavirus24 | they don't do /foo/bar/bogus/**/*.something | 20:56 |
| sigmavirus24 | so perhaps the answer is to eliminate the /*.log from the end | 20:56 |
| sigmavirus24 | probably unnecessary anyway | 20:56 |
| jhesketh | sigmavirus24: well then you'll need one * | 20:57 |
| sigmavirus24 | jhesketh: I'm considering updating it to just be /openstack/log/** | 20:57 |
| sigmavirus24 | wouldn't that recursively copy the files in those directories? | 20:57 |
| *** tsg has quit IRC | 20:57 | |
| sigmavirus24 | Also, I wonder if some kind of tests around ANT's parsing of globs would be good to have at the project-config gate | 20:58 |
| jhesketh | sigmavirus24: hmm, I /think/ you can just point it at a dir and it'll copy all the contents, but I'm not sure now | 20:58 |
| *** tsg has joined #openstack-infra | 20:58 | |
| sigmavirus24 | jhesketh: I'd expect that's how the docs-draft and docs-publish jobs are working | 20:58 |
| sigmavirus24 | jhesketh: https://github.com/openstack-infra/project-config/blob/master/jenkins/jobs/macros.yaml#L596 | 20:59 |
| *** ddieterly has quit IRC | 20:59 | |
| sigmavirus24 | I should probably check for documentation of the macros too just to make sure keep_hierarchy means what we think it does | 20:59 |
| *** ddieterly has joined #openstack-infra | 20:59 | |
| *** tsg_ has joined #openstack-infra | 21:00 | |
| *** gyee has joined #openstack-infra | 21:02 | |
| openstackgerrit | Merged openstack-infra/elastic-recheck: Add query for neutron DBDeadlock bug 1426543 https://review.openstack.org/159993 | 21:03 |
| *** tsg has quit IRC | 21:03 | |
| openstack | bug 1426543 in OpenStack Compute (nova) "Spike in DBDeadlock errors in update_floatingip_statuses since 2/27" [Undecided,New] https://launchpad.net/bugs/1426543 | 21:03 |
| *** achanda has joined #openstack-infra | 21:03 | |
| *** pelix has quit IRC | 21:04 | |
| *** achanda has quit IRC | 21:05 | |
| *** Ryan_Lane has quit IRC | 21:06 | |
| *** marun has quit IRC | 21:06 | |
| *** MaxV has quit IRC | 21:06 | |
| flashgordon | mordred: looking forward to what https://github.com/openstack/openstack/graphs/commit-activity looks like in a week from now | 21:10 |
| *** achanda has joined #openstack-infra | 21:10 | |
| flashgordon | with the new definition in place | 21:10 |
| *** jedimike has quit IRC | 21:11 | |
| jeblair | reed, mtreinish: if there are any metrics systems that need to discount reviews that are not active, please do so in those metrics systems | 21:11 |
| jeblair | reed, mtreinish: do not count on something like the abandon old reviews script to abandon changes | 21:11 |
| sigmavirus24 | jhesketh, anteaya: so on my aio from a regular user to another regular user, I can scp /openstack/log/**/*.log without issue | 21:11 |
| sigmavirus24 | It must just be an Ant grammar issue | 21:11 |
| jeblair | reed, mtreinish: i think it is very rude to automatically abandon someone's change, particularly since there is no longer any need to do so with new dashboards and such | 21:11 |
| jeblair | reed, mtreinish: the facility for core reviewers to personally abandon changes was set up so that core reviewers would have discrection to personally abandon changes that were no longer relevant | 21:12 |
| *** ldnunes has quit IRC | 21:12 | |
| jeblair | reed, mtreinish: using that as the basis of an automated system was not our intent | 21:12 |
| Ng | mordred: do you happen to have easy access to the contents of an OnMetal config drive that I can poke around with locally? | 21:12 |
| *** abhirc has quit IRC | 21:13 | |
| jeblair | reed, mtreinish: particularly with nova saying things like "-2 this change can't land until liberty opens", that seems particularly incompatible with abandoning reviews | 21:13 |
| mordred | Ng: I do not | 21:13 |
| mordred | Ng: but! you don't need one - all yo uneed is a normal rax vm config drive | 21:13 |
| mordred | Ng: it's rolled to their vm nova too | 21:14 |
| *** thedodd has joined #openstack-infra | 21:14 | |
| Ng | can I pinch a copy of a vm config drive off someone? I don't currently have any rax stuff | 21:15 |
| JayF | Ng: I am about to board a plane, but if I have an onmetal box booted I can give you a configdrive from it, sec | 21:16 |
| sdague | anyone understand why jenkins -1ed this - https://review.openstack.org/#/c/159825, because the whole stack just got pushed and the patches below it seem fine | 21:16 |
| Ng | JayF: awesome! thanks | 21:16 |
| JayF | Ng: you really just want vendor_data['network'], I'm guessing? | 21:17 |
| morganfainberg | sdague, the parent failed to merge? | 21:17 |
| Ng | JayF: that's the bit I'm most interested in, yeah, but I wouldn't object to the whole thing, if that's not problematic | 21:17 |
| sdague | right, which it didn't | 21:18 |
| JayF | Ng: just trying to optimize for time | 21:18 |
| JayF | literally my plane boards in <5m | 21:18 |
| morganfainberg | sdague, yeah chasing the chain, wow, weird. | 21:18 |
| sdague | yep | 21:18 |
| Ng | JayF: I'm going to get evil looks if I don't stop working and sleep soon, so I am also fine with trying again on Monday :) | 21:19 |
| *** e0ne_ is now known as e0ne | 21:19 | |
| JayF | Ng: I don't have a VM booted /with configdrive/. Sorry. | 21:19 |
| Ng | JayF: np, thanks anyway! | 21:19 |
| morganfainberg | sdague, jgit being picky about something? | 21:19 |
| JayF | Ng: feel free to ping me on monday and I can help you out :) | 21:19 |
| morganfainberg | sdague, but usually thats the blue bird of merge conflict. | 21:19 |
| morganfainberg | sdague, hmm... | 21:19 |
| morganfainberg | sdague, the parent also claimed it failed to merge, but jenkins +1'd it | 21:20 |
| morganfainberg | Patch Set 4: Verified+1This change depends on a change that failed to merge. | 21:20 |
| sdague | oh, that's even weirder | 21:20 |
| morganfainberg | sdague, this looks like maybe a wire or three got crossed somewhere | 21:21 |
| sdague | yeh, these aren't running tests at the moment | 21:21 |
| morganfainberg | because 2 layers up in the chain i don't see the failure | 21:21 |
| openstackgerrit | Ian Cordasco proposed openstack-infra/project-config: Fix glob pattern for os-ansible-logs publisher https://review.openstack.org/160006 | 21:22 |
| rackertom | sigmavirus24: Would you like me to submit a fix for this? I suspected the /*.log would be trouble | 21:22 |
| sigmavirus24 | rackertom: https://review.openstack.org/#/c/160006/ | 21:22 |
| sdague | I just rebased again, and it still got marked the same way | 21:22 |
| openstackgerrit | Joe Gordon proposed openstack-infra/elastic-recheck: Make test_queries runnable as gating test https://review.openstack.org/150186 | 21:22 |
| openstackgerrit | Joe Gordon proposed openstack-infra/elastic-recheck: Add launchpad test to test_queries https://review.openstack.org/155138 | 21:22 |
| sdague | hmph | 21:22 |
| morganfainberg | sdague, ok that is just wierd. | 21:22 |
| *** fungi has quit IRC | 21:23 | |
| rackertom | sigmavirus24: 👍 Nice | 21:23 |
| morganfainberg | sdague, maybe a cranky jenkins? | 21:24 |
| sigmavirus24 | morganfainberg: jenkins? cranky? nevvvvverrrrr | 21:24 |
| morganfainberg | sdague, the +1 with the failure notice is weirding me out | 21:24 |
| morganfainberg | oh | 21:24 |
| sdague | yeh, something is pretty wrong. oh well. | 21:24 |
| morganfainberg | sdague, wow | 21:24 |
| morganfainberg | sdague, look at it, you are getting 2 notices | 21:24 |
| morganfainberg | a -1 failure, and a +1 failure on the parent | 21:25 |
| sdague | yeh | 21:25 |
| sdague | and I *just* rebased this stack | 21:25 |
| morganfainberg | yep i see that | 21:25 |
| mtreinish | morganfainberg: I would just blame jgit... | 21:25 |
| mtreinish | even if it's unrelated :) | 21:25 |
| *** fungi has joined #openstack-infra | 21:25 | |
| morganfainberg | mtreinish, haha ok well sure. | 21:25 |
| sdague | mtreinish: except, it's a cleanly rebased patch tree | 21:26 |
| *** stevemar has quit IRC | 21:26 | |
| *** dustins has quit IRC | 21:28 | |
| *** sambetts has quit IRC | 21:28 | |
| morganfainberg | sdague, i think you have 2 things getting triggered somehow. and it's causing something to lose it's mind. | 21:28 |
| * mordred loses his mind | 21:28 | |
| morganfainberg | unfortunately, i don't think either of us can see that deep into -infra logging of things like jenkins | 21:28 |
| reed | jeblair, I don't have a strong opinion on that. I think it's also rude to send a patch and leave it languish there... I don't argue in favor of auto-abandon | 21:29 |
| morganfainberg | sdague, at least i'm at my limit of what i could see ;) | 21:29 |
| reed | jeblair, maybe auto-abandon should also come with an offer to mentor? | 21:29 |
| jeblair | reed: we have a lot of tools now to help prioritize workflow now, including gerrit dashboards which can automatically filter out patches which people feel are inactive | 21:30 |
| reed | jeblair, something like "This patch has no chances of merging as is and if you want, you can reach out to me and I'll help you to fix it" | 21:30 |
| jeblair | reed: i think that if nova's workflow is now, literally, "go away for 3 months", we can hardly blame people for leaving patches sitting outstanding for 3 months | 21:30 |
| *** sambetts has joined #openstack-infra | 21:30 | |
| jeblair | reed: i think it's fine to leave a message like that, but i don't think it needs to be abandoned. my oldest patch is from july. it should still be revised and merged. i don't think anyone thinks it's rude that hasn't happened yet. | 21:31 |
| jeblair | (if anyone did, they could, of course, fix up the outstanding comment) | 21:31 |
| *** baoli has quit IRC | 21:31 | |
| *** e0ne has quit IRC | 21:31 | |
| reed | jeblair, maybe a better workflow is to collect the list of patches that need attention and make a call for actions instead | 21:31 |
| jeblair | (and i don't think it's bothering anyone that it's still sitting there, except me, which is as it should be :) | 21:31 |
| *** e0ne has joined #openstack-infra | 21:32 | |
| reed | I think we should keep the conversation on the list though, this is important knowledge we should capture... | 21:32 |
| jeblair | reed: that seems good too, except we should still account for the fact that maybe the action for some of them is wait 3 months until nova is willing to talk to them again :) | 21:32 |
| reed | jeblair, ACK... although that workflow seems flawed to me :) | 21:32 |
| jeblair | reed: i agree, i think i replied to the list with many of the points i made here | 21:33 |
| jeblair | reed: no argument there, but then, that's what those two threads are about, eh? :) | 21:33 |
| reed | jeblair, neat, thanks. I didn't read that message yet | 21:33 |
| *** unicell has quit IRC | 21:33 | |
| *** unicell has joined #openstack-infra | 21:33 | |
| *** marun has joined #openstack-infra | 21:34 | |
| jeblair | sdague, morganfainberg: https://review.openstack.org/#/c/157080/ recently merged but is not active because we have not restarted zuul to pick it up | 21:35 |
| *** yamamoto has joined #openstack-infra | 21:35 | |
| sdague | jeblair: ok | 21:35 |
| jeblair | sdague, morganfainberg: it _may_ have a bearing on the behavior you are seeing; i think the best way to proceed is to see if we see something like that again with that patch in place | 21:35 |
| morganfainberg | jeblair, ah | 21:35 |
| morganfainberg | jeblair, cool. | 21:35 |
| jeblair | and zuul is actually quiet enough that i think i could do a restart now | 21:36 |
| * morganfainberg quickly queues 1 billion things up *shifty eyes* | 21:36 | |
| * jeblair restarts zuul | 21:36 | |
| morganfainberg | friday getting close to end of day most everywhere but pacific coast... yeah it's probably quieted down | 21:36 |
| morganfainberg | jeblair, ooh i need to respond to your zuul thread on the ML. | 21:37 |
| * morganfainberg just remembered | 21:37 | |
| morganfainberg | mordred, so talked with jamielennox about DNS serv records... we just need to find the right place to stick it. but i think everyone would like that nice little QOL change. | 21:38 |
| morganfainberg | mordred, fyi | 21:38 |
| mordred | morganfainberg: yay! | 21:38 |
| jeblair | sdague, morganfainberg: zuul is restarted now and is running master; let me know if you see it again | 21:38 |
| *** tonytan4ever has quit IRC | 21:38 | |
| jeblair | morganfainberg: oh yeah, dns srv++ | 21:39 |
| *** yamamoto has quit IRC | 21:39 | |
| *** baoli has joined #openstack-infra | 21:39 | |
| morganfainberg | jeblair, i think it can go in the discover code. *think* | 21:39 |
| morganfainberg | but i need to go rip that apart cause that is some of the crazy dense code in ksc that makes up session | 21:40 |
| morganfainberg | but thats cause session is.. well trying to make other clients less complex. | 21:40 |
| jeblair | marun, flashgordon: zuul is restarted; i left a comment indicating so on 159582 | 21:40 |
| mtreinish | jeblair: so something interesting I see post-restart, on the zuul status page I see 143159,25 queued on check, but there are already results for that reported (and no recheck) | 21:41 |
| *** sigmavirus24 is now known as sigmavirus24_awa | 21:42 | |
| *** jhutton has quit IRC | 21:42 | |
| flashgordon | jeblair: thanks! | 21:42 |
| jeblair | mtreinish: ah, i think my script to save queues has a defect, thanks, will fix. | 21:43 |
| *** sigmavirus24_awa is now known as sigmavirus24 | 21:43 | |
| *** achanda has quit IRC | 21:43 | |
| *** achanda has joined #openstack-infra | 21:44 | |
| *** mfink_ has quit IRC | 21:44 | |
| *** otter768 has joined #openstack-infra | 21:44 | |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul: Update zuul-changes to use the enqueue command https://review.openstack.org/160015 | 21:46 |
| *** tiswanso_ has joined #openstack-infra | 21:46 | |
| marun | jeblair: great! | 21:47 |
| *** rmcall has joined #openstack-infra | 21:48 | |
| * dims_ reading scrollback to see if i can find why zuul page is all "queued" | 21:49 | |
| *** otter768 has quit IRC | 21:49 | |
| *** prad has quit IRC | 21:49 | |
| dims_ | ah. "zuul is restarted" paydirt | 21:49 |
| *** tiswanso has quit IRC | 21:50 | |
| *** amitgandhinz has quit IRC | 21:51 | |
| *** radez is now known as radez_g0n3 | 21:51 | |
| *** tiswanso_ has quit IRC | 21:51 | |
| *** aysyd has quit IRC | 21:51 | |
| *** dims_ is now known as dimsum__ | 21:53 | |
| jeblair | dimsum__: thanks, i think a bug was introduced in a recent change, tracking it down | 21:54 |
| dimsum__ | jeblair: cool | 21:55 |
| *** ghostpl_ has joined #openstack-infra | 21:55 | |
| *** sabeen1 has quit IRC | 21:57 | |
| jeblair | jhesketh, clarkb: ping | 22:00 |
| anteaya | jhesketh: is saturdaying with a meeting, pleia2 is eating a sandwich | 22:00 |
| anteaya | not sure what I can do but I'm here | 22:00 |
| jeblair | anteaya: i want to ask clarkb what motivated him to write 157592 | 22:01 |
| jeblair | anteaya: it appears related to the bug we are seeing, but it does not mention why it should suddenly be necessary | 22:01 |
| *** barra204_ has joined #openstack-infra | 22:03 | |
| anteaya | I think it was due to a type of error the swift logs were throwing | 22:03 |
| *** shakamunyi has quit IRC | 22:03 | |
| *** mbacchi has quit IRC | 22:03 | |
| *** amitgandhinz has joined #openstack-infra | 22:04 | |
| *** mattfarina has quit IRC | 22:04 | |
| *** chuckC_ has joined #openstack-infra | 22:06 | |
| *** amitgandhinz has quit IRC | 22:07 | |
| *** barra204_ has quit IRC | 22:07 | |
| *** amitgandhinz has joined #openstack-infra | 22:07 | |
| *** barra204_ has joined #openstack-infra | 22:08 | |
| *** annegentle has quit IRC | 22:10 | |
| *** amitgandhinz has quit IRC | 22:10 | |
| *** amitgandhinz has joined #openstack-infra | 22:10 | |
| anteaya | jeblair: found it: 2015-02-19T23:14:09 <clarkb> AttributeError: 'int' object has no attribute 'format' | 22:10 |
| anteaya | http://eavesdrop.openstack.org/irclogs/%23openstack-infra/%23openstack-infra.2015-02-19.log | 22:11 |
| *** annegentle has joined #openstack-infra | 22:11 | |
| *** rfolco has quit IRC | 22:13 | |
| *** mjturek1 has quit IRC | 22:17 | |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul: Fix int conversion in swift https://review.openstack.org/160023 | 22:17 |
| *** cody-somerville_ has joined #openstack-infra | 22:19 | |
| *** cody-somerville_ has quit IRC | 22:19 | |
| *** cody-somerville_ has joined #openstack-infra | 22:19 | |
| *** cody-somerville has quit IRC | 22:19 | |
| jeblair | clarkb, jhesketh: ^ i am manually applying that as well as 157592 | 22:19 |
| jeblair | clarkb, jhesketh: the change in puppet-zuul to start specifying that value in the config file triggered the bug, as that value is read as a str with no conversion, while defaults in code are ints | 22:20 |
| *** thedodd has quit IRC | 22:21 | |
| anteaya | and tests are testing | 22:23 |
| anteaya | yay | 22:23 |
| *** jamielennox is now known as jamielennox|away | 22:23 | |
| anteaya | sdague: thanks | 22:24 |
| *** bswartz has quit IRC | 22:25 | |
| *** _shaps__ has joined #openstack-infra | 22:29 | |
| openstackgerrit | Merged openstack-infra/project-config: Revert "Revert "Add conditional skips for nova jobs"" https://review.openstack.org/159582 | 22:29 |
| *** e0ne has quit IRC | 22:30 | |
| openstackgerrit | Merged openstack-infra/project-config: Change default behaviour of -nv https://review.openstack.org/159916 | 22:30 |
| *** jamielennox|away is now known as jamielennox | 22:30 | |
| *** nelsnels_ has joined #openstack-infra | 22:31 | |
| *** pc_m has quit IRC | 22:31 | |
| *** nelsnelson has quit IRC | 22:32 | |
| mrmartin | re | 22:33 |
| *** markvan has quit IRC | 22:33 | |
| *** markvan_ is now known as markvan | 22:33 | |
| clarkb | jeblair: catching up | 22:34 |
| mrmartin | pleia2, fungi, jeblair: please check this review: https://review.openstack.org/#/c/159426/ I need an approval, because the scheduled tasks are broken on groups.o.o portal without this patch. thnx. | 22:34 |
| clarkb | mordred: I got greghaynes to figure this thing out | 22:34 |
| *** markvan_ has joined #openstack-infra | 22:35 | |
| clarkb | jeblair: gah sorry, | 22:35 |
| clarkb | jeblair: so we fixed it for the layout.yaml ints but not for ini strings to ints | 22:35 |
| greghaynes | yea, its part of a larger deal where we need to make source-repositories a bit more featureful | 22:36 |
| clarkb | mordred: greghaynes says that we should do the caching late with a bind mounted dib cache in the image | 22:36 |
| flashgordon | jeblair: so https://review.openstack.org/#/c/159582/ landed, but it will take some time for it to come into effect correct? | 22:36 |
| greghaynes | and also get rid of the you copying the whole source-repositories into an element *shakes fist* | 22:36 |
| flashgordon | anteaya: https://review.openstack.org/#/c/160027/ look at mellanox CI | 22:37 |
| flashgordon | that voted way too quick to do anything | 22:37 |
| *** amitgandhinz has quit IRC | 22:37 | |
| clarkb | flashgordon: has anyone writtn the change to put aiopcpu in neutron's experimental queue yet? | 22:38 |
| *** amitgandhinz has joined #openstack-infra | 22:38 | |
| flashgordon | clarkb: oh no, I'll do that now | 22:38 |
| *** jamielennox is now known as jamielennox|away | 22:39 | |
| jeblair | flashgordon: you can check the 'last reconfigured' time at the bottom of the zuul status page | 22:39 |
| mordred | clarkb: I'm just going to trust greghaynes | 22:39 |
| reed | jeblair, I found someone who had at least two patches automatically closed ... at least I have someone to talk to and ask for personal opinions | 22:39 |
| flashgordon | jeblair: neato, thanks | 22:39 |
| greghaynes | mordred: famous last words | 22:39 |
| reed | jeblair, https://review.openstack.org/#/q/owner:%22Mh+Raies+%253Craiesmh08%2540gmail.com%253E%22,n,z | 22:39 |
| jeblair | flashgordon: i think it should happen within maybe 30 minutes based on current ansible puppet performance? | 22:39 |
| reed | jeblair, quite a lot of auto-abandoned actually, all those on Dec 5 | 22:40 |
| flashgordon | jeblair: thanks | 22:40 |
| *** jamielennox|away is now known as jamielennox | 22:40 | |
| *** sarob has joined #openstack-infra | 22:40 | |
| openstackgerrit | Joe Gordon proposed openstack-infra/project-config: Add aiopcpu neutron job to neutron experimental https://review.openstack.org/160028 | 22:41 |
| flashgordon | clarkb: ^ | 22:41 |
| *** chuckC_ has quit IRC | 22:41 | |
| clarkb | flashgordon: thanks | 22:41 |
| *** esker has quit IRC | 22:41 | |
| jeblair | reed: i know what would happen to me if one of mine were auto-abandoned. i would never notice. until, maybe, someday, something happened and i thought to myself, "i thought i fixed that". and after a couple of hours of digging, i would find out what happened, and i would be furious. i would probably never contribute to the project again. | 22:42 |
| anteaya | flashgordon: well that and there are no job logs linked | 22:42 |
| *** amitgandhinz has quit IRC | 22:42 | |
| anteaya | flashgordon: it didn't vote it commented | 22:42 |
| openstackgerrit | Akshay Chhajed proposed openstack-infra/project-config: Solum: Modify requirement flag https://review.openstack.org/160030 | 22:42 |
| anteaya | omr | 22:43 |
| reed | jeblair, yeah... but I wonder how much of that feeling is the amount of comments on the reviews or the auto-abandon... if you see some of those reviews, at some point he failed to respond | 22:43 |
| anteaya | flashgordon: can't seem to find anyone associated with that system in channel | 22:43 |
| anteaya | flashgordon: did they ask nova's permission to comment on nova patches? | 22:43 |
| reed | jeblair, on one instead, after he worked for a long time, his patch ended up being made obsolete by someone else's contribution | 22:43 |
| jeblair | reed: yeah, not saying my case is universal, just giving you an anecdata point :) | 22:43 |
| flashgordon | anteaya: not that I know of, but I may have missed that | 22:44 |
| reed | jeblair, like this one: multiple +1, https://review.openstack.org/#/c/98974/ then at the end 'fix the commit message' and silence after that | 22:44 |
| *** akshayc has joined #openstack-infra | 22:45 | |
| reed | or this one https://review.openstack.org/#/c/89472/ lots of work then "oops, I forgot you did this already here and my patch already merged... please abandon it" | 22:45 |
| flashgordon | anteaya: looks fairly sporadic though | 22:45 |
| zaro | clarkb: try without being optional: https://gerrit-review.googlesource.com/#/c/65530 | 22:45 |
| jeblair | reed: yeah, someone probably should have just fixed the commit message, huh? | 22:46 |
| reed | jeblair, yeah... there is a trove of anecdotes in there :) | 22:46 |
| flashgordon | anteaya: http://paste.openstack.org/show/183223 | 22:46 |
| anteaya | flashgordon: what looks sporadic? that account commenting on nova patches? | 22:46 |
| jeblair | reed: here's another anecdote: https://review.openstack.org/#/c/142589/ | 22:46 |
| jeblair | reed: jaypipes has, apparently, abandoned his work despite it just needing a minor correction | 22:46 |
| jeblair | reed: because the patch is still there, i know about it and can see the state. i'm about to just pick up his work and fix it | 22:47 |
| anteaya | flashgordon: that is just on the openstack/nova repo, correct? | 22:47 |
| reed | jeblair, yes, that behavior is to be encouraged I think | 22:47 |
| jeblair | reed: (even though, tomorrow, it would be eligible for auto-abanonment by the commonly-used script) | 22:47 |
| clarkb | zaro: thanks, we'll hvae to see what they say | 22:47 |
| flashgordon | not filtered so all repos not sure if that is just nova or not | 22:48 |
| flashgordon | anteaya: using my last comment script | 22:48 |
| reed | jeblair, i'm leaning toward your position that we should not self-abandon but mark the items for review by humans | 22:48 |
| anteaya | flashgordon: well their wikipage just identifies them as testing neutron: https://wiki.openstack.org/wiki/ThirdPartySystems/Mellanox_CI | 22:48 |
| anteaya | flashgordon: ah okay, well it is a handy script, but they are allowed to comment on neutron | 22:48 |
| reed | sort of like "flagging for action: do we still need this? what does it take to finish it?" | 22:49 |
| anteaya | flashgordon: and vote on neutron: https://review.openstack.org/#/admin/groups/510,members | 22:49 |
| reed | food for thoughts... I need to finish the newsletter now | 22:49 |
| jeblair | reed: omg i would love tags in gerrit for something like this | 22:49 |
| anteaya | flashgordon: I'm guessing that somehow they changed their config and got commenting on nova patches | 22:49 |
| flashgordon | anteaya: just thought it was odd they have no-op comments on nova patches | 22:50 |
| flashgordon | its not a big deal though | 22:50 |
| anteaya | flashgordon: it is, thanks for bringing it to my attention | 22:50 |
| anteaya | flashgordon: how would it sound if I post to the third-party-announce email list telling them to stop commenting on nova patches and giving them until monday to comply | 22:51 |
| *** Ryan_Lane has joined #openstack-infra | 22:51 | |
| *** abhirc has joined #openstack-infra | 22:51 | |
| flashgordon | anteaya: sounds reasonable to me. no-op comments are sort of silly | 22:51 |
| flashgordon | and there is no way this is testing anything meaningful | 22:51 |
| anteaya | agreed | 22:52 |
| *** kgiusti has left #openstack-infra | 22:53 | |
| *** mattfarina has joined #openstack-infra | 22:55 | |
| *** mattfarina has quit IRC | 22:55 | |
| *** ddieterly has quit IRC | 22:55 | |
| clarkb | jeblair: so the failed test was testing that abandoning a change works correctly. We get http://paste.openstack.org/show/183224/ I think the exception means the job isn't cancelled so gets status SUCCESS | 22:56 |
| clarkb | jeblair: looking into how that exception could occur now | 22:56 |
| *** annegentle has quit IRC | 22:56 | |
| openstackgerrit | James E. Blair proposed stackforge/gertty: Add some INFO level log messages https://review.openstack.org/143165 | 22:57 |
| openstackgerrit | James E. Blair proposed stackforge/gertty: Add INFO log level potential with --verbose flag https://review.openstack.org/143144 | 22:57 |
| openstackgerrit | James E. Blair proposed stackforge/gertty: Speed up the toggling of reviewed/hidden changes https://review.openstack.org/142590 | 22:57 |
| openstackgerrit | James E. Blair proposed stackforge/gertty: Release DB session thread lock earlier in syncs https://review.openstack.org/142589 | 22:57 |
| jeblair | clarkb: cool, i have to run again, sorry :( | 22:57 |
| clarkb | jeblair: I think the race is that we abandon the job before any job statuses are passed in | 22:57 |
| *** hichtakk has joined #openstack-infra | 22:57 | |
| clarkb | jeblair: no problem I think we can fix this by just setting the manager to None upfront | 22:57 |
| *** barra204_ has quit IRC | 22:57 | |
| *** abhirc has quit IRC | 22:58 | |
| jeblair | mordred: oh, check out the tip of that branch (143165) and see if it feels more responsive (and if not, try to note what's going on when it isn't) | 22:58 |
| clarkb | so I am writing that change now | 22:58 |
| anteaya | flashgordon: http://lists.openstack.org/pipermail/third-party-announce/2015-February/000159.html | 22:58 |
| anteaya | flashgordon: let me know on monday if you are still seeing it | 22:58 |
| *** sarob has quit IRC | 22:59 | |
| anteaya | and my evening is arriving shortly | 23:00 |
| anteaya | have a good weekend | 23:00 |
| flashgordon | anteaya: o/ | 23:01 |
| *** mriedem has quit IRC | 23:01 | |
| *** sarob has joined #openstack-infra | 23:01 | |
| *** akshayc has left #openstack-infra | 23:01 | |
| *** jgrimm is now known as zz_jgrimm | 23:01 | |
| *** unicell has quit IRC | 23:02 | |
| *** unicell1 has joined #openstack-infra | 23:02 | |
| *** dmsimard is now known as dmsimard_away | 23:02 | |
| *** jamielennox is now known as jamielennox|away | 23:06 | |
| *** annegentle has joined #openstack-infra | 23:06 | |
| openstackgerrit | Sean Roberts proposed openstack-infra/system-config: patch to create test-certification-standards ilist https://review.openstack.org/159995 | 23:06 |
| *** sarob has quit IRC | 23:07 | |
| *** _shaps__ has quit IRC | 23:08 | |
| *** andreykurilin_ has joined #openstack-infra | 23:12 | |
| openstackgerrit | jose-idar proposed openstack-infra/project-config: Adding pep8 gate check to Opencafe https://review.openstack.org/159642 | 23:12 |
| mordred | jeblair: I look forward to such things | 23:14 |
| *** achanda has quit IRC | 23:16 | |
| *** doug-fish has left #openstack-infra | 23:18 | |
| *** dangers is now known as dangers_away | 23:18 | |
| *** dimtruck is now known as zz_dimtruck | 23:19 | |
| *** achanda has joined #openstack-infra | 23:20 | |
| *** nelsnels_ has quit IRC | 23:20 | |
| *** mrmartin has quit IRC | 23:22 | |
| *** derekh has joined #openstack-infra | 23:23 | |
| flashgordon | fungi: looking at zuul for nova patch 160027 | 23:31 |
| flashgordon | which only touches docs/ so it is only running pep8 and docs | 23:32 |
| flashgordon | and the expected time says 57 minutes which sounds closer to the expected time for the full set of tests | 23:32 |
| *** gordc has quit IRC | 23:32 | |
| *** bswartz has joined #openstack-infra | 23:32 | |
| *** koolhead17 has quit IRC | 23:34 | |
| *** dims_ has joined #openstack-infra | 23:37 | |
| *** dimsum__ has quit IRC | 23:38 | |
| *** amitgandhinz has joined #openstack-infra | 23:38 | |
| *** dims_ has quit IRC | 23:39 | |
| *** dimsum__ has joined #openstack-infra | 23:39 | |
| wznoinsk | asselin: are you around? | 23:43 |
| asselin | wznoinsk, hi, yes | 23:43 |
| *** amitgandhinz has quit IRC | 23:43 | |
| wznoinsk | hi there, I maintain Intel Networking CI (https://wiki.openstack.org/wiki/ThirdPartySystems/Intel-Networking-CI), I haven't requested access to Third Party CI group in Gerrit, in what circumstances I may need this membership? | 23:45 |
| *** otter768 has joined #openstack-infra | 23:45 | |
| *** zz_dimtruck is now known as dimtruck | 23:46 | |
| asselin | wznoinsk, all, I can do that now | 23:46 |
| wznoinsk | yes, please | 23:47 |
| asselin | wznoinsk, done https://review.openstack.org/#/admin/groups/270,members | 23:47 |
| asselin | Intel Networking CIopenstack-networking-ci@intel.com | 23:47 |
| wznoinsk | thanks, when I may need to send email via gerrit (not sure what that's used for to be completely honest)? | 23:48 |
| wznoinsk | is it to request code review and such? | 23:49 |
| asselin | wznoinsk, no, it just mean when you CI posts a comment, then Gerrit won't send an e-mail to all the reviews | 23:50 |
| *** otter768 has quit IRC | 23:50 | |
| openstackgerrit | Clark Boylan proposed openstack-infra/zuul: Set __gearman_mananger on builds at launch time https://review.openstack.org/160050 | 23:50 |
| clarkb | jeblair: ^ Ithink thats the fix | 23:50 |
| asselin | *reviewers | 23:50 |
| asselin | i.e. stops the spam :) | 23:50 |
| wznoinsk | good, thanks | 23:51 |
| asselin | wznoinsk, np | 23:51 |
| *** hockeynut has quit IRC | 23:51 | |
| *** hockeynut has joined #openstack-infra | 23:52 | |
| *** chuckC_ has joined #openstack-infra | 23:52 | |
| wznoinsk | speaking of permissions to comment on projects (ie. mellanox on nova), I wasn't aware you have to get permissions to comment on projects (or it's just our special Nova has that) ? | 23:53 |
| dvorak | I've got a review that's got +1 workflow label, but isn't merging, and there are no gate tests for this project | 23:54 |
| dvorak | any idea what might be going on? https://review.openstack.org/#/c/160024/ | 23:54 |
| dvorak | I don't see anything on zuul for it | 23:54 |
| wznoinsk | I would like to avoid breaking any rules, good practices before I have our Compute (numa) CI commenting on NOva soon | 23:54 |
| wznoinsk | asselin: <- | 23:54 |
| *** jamesmcarthur has joined #openstack-infra | 23:55 | |
| asselin | wznoinsk, the rules should be here under "Requirements" http://ci.openstack.org/third_party.html | 23:55 |
| asselin | wznoinsk, I looked quickly and don't see a mention of the rule to get permission on a project. perhaps that needs to be added so it's clear. | 23:56 |
| asselin | wznoinsk, I have to go now, have a nice weekend. | 23:56 |
| wznoinsk | yes, didn't recall these when I was following that guide, I'll request them beforehand, | 23:57 |
| wznoinsk | have a good one asselin | 23:57 |
| *** baoli has quit IRC | 23:57 | |
| clarkb | dvorak: does the project not have any gate tests? if not then nothing can merge, you will need to add some tests. there is a noop test that can be used | 23:59 |
| clarkb | hrm looks like it has tests configured | 23:59 |
| dvorak | it's not my project, but this worked earlier today | 23:59 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!
