| anteaya | mattoliverau: do your best, if your information needs correcting someone will step in | 00:00 |
|---|---|---|
| *** doddstack has quit IRC | 00:00 | |
| anteaya | mattoliverau: and thanks | 00:00 |
| fungi | mordred: on https://review.openstack.org/#/c/90842/1/modules/openstack_project/files/gerrit/acls/openstack-dev/hacking.config what's the reasoning behind removing create on refs/*? i think that's being used by some project teams to create branches | 00:00 |
| * anteaya tapers down | 00:00 | |
| mordred | fungi: ah - basically, I removed create from things because we had it in place in support of creating tags everywhere and is not actually needed for that purpose anymore | 00:01 |
| mattoliverau | anteaya: thanks, all I can do it help anyone the best I can and let them know that need to take it with a grain of salt if it's something I'm not entirely sure of :) You go sleep! you deserve it! | 00:01 |
| *** jhesketh has quit IRC | 00:01 | |
| mordred | fungi: it's entirely possible that tere are legitimate reasons for that line to be places | 00:01 |
| *** jhesketh has joined #openstack-infra | 00:02 | |
| fungi | mattoliverau: the easy answer is to ask people why they're doing stuff on the project vacation week ;) | 00:03 |
| *** aysyd has joined #openstack-infra | 00:03 | |
| mattoliverau | fungi: lol, true :) | 00:04 |
| jeblair | mordred: i just merged a change to sandbox | 00:04 |
| jeblair | mordred: it hasn't shown up on the apache replica yet | 00:05 |
| mordred | jeblair: I dont' see any issuse in the log | 00:05 |
| *** SumitNaiksatam_ has joined #openstack-infra | 00:06 | |
| *** jhesketh has quit IRC | 00:07 | |
| jeblair | mordred: all 5 git servers are updated but not the local replica | 00:07 |
| mordred | oh | 00:07 |
| mordred | well that's strange | 00:07 |
| *** SumitNaiksatam has quit IRC | 00:07 | |
| *** lcostantino has joined #openstack-infra | 00:07 | |
| *** jhesketh_ has joined #openstack-infra | 00:07 | |
| jeblair | mordred: becaus it's replicating to the wrong place :( | 00:08 |
| fungi | gah | 00:08 |
| jeblair | patch incoming | 00:08 |
| mordred | I thnk I may have just realized what you're saying and I'm placing my heands on my face | 00:08 |
| *** atiwari has quit IRC | 00:08 | |
| fungi | i didn't even think about the replication config | 00:08 |
| jeblair | mordred: that's a good place for them. my hands will be doing something similar soon | 00:08 |
| mordred | also - by the by - jeepyb is looking at localhost for update_bug and blueprint | 00:08 |
| fungi | we moved that entire tree | 00:08 |
| *** asettle has joined #openstack-infra | 00:08 | |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/config: Move configured gerrit replica to /opt/lib/git https://review.openstack.org/90856 | 00:09 |
| *** SumitNaiksatam_ is now known as SumitNaiksatam | 00:09 | |
| fungi | mordred: update_* are looking at localhost copies of the git repos, or for the db? | 00:09 |
| mordred | db | 00:10 |
| fungi | if the latter, we probably want them to use the jeepyb.gerritdb module | 00:10 |
| openstackgerrit | A change was merged to openstack-infra/config: Increase replication delay to 1 second https://review.openstack.org/90850 | 00:10 |
| openstackgerrit | Monty Taylor proposed a change to openstack-infra/jeepyb: Get the db host from the gerrit config too https://review.openstack.org/90857 | 00:10 |
| jeblair | restarting gerrit | 00:10 |
| mordred | fungi: ^^ | 00:10 |
| mordred | jeblair: ++ | 00:10 |
| jeblair | mordred: didn't i already do that? | 00:10 |
| *** pcrews has quit IRC | 00:11 | |
| mordred | jeblair: you did it for a different thing I thought | 00:11 |
| *** loquacities has quit IRC | 00:11 | |
| jeblair | mordred: we'll see when gerrit comes back up. :) | 00:11 |
| mordred | :) | 00:11 |
| mordred | jeblair: you did it for manage_projects | 00:11 |
| *** bradm has quit IRC | 00:11 | |
| *** loquacities has joined #openstack-infra | 00:11 | |
| mordred | maybe? | 00:11 |
| * mordred headdesks for good measure | 00:12 | |
| jeblair | mordred: you are right. that is, for some reason, different. it doesn't seem to use jeepydb.gerritdb | 00:12 |
| mordred | yeah | 00:13 |
| mordred | I thnk we keep ignoring it and hoping it'll go away | 00:13 |
| mordred | that said - your change isn't landed iether | 00:13 |
| mordred | I think I should go in and approve today's outstanding changes that at least one of us as +2'd | 00:13 |
| mordred | oh! | 00:13 |
| mordred | a failure | 00:13 |
| mordred | with the jenkins | 00:13 |
| * mordred fix jeblair patch | 00:14 | |
| *** gokrokve has joined #openstack-infra | 00:14 | |
| openstackgerrit | Monty Taylor proposed a change to openstack-infra/jeepyb: Use configured mysql host https://review.openstack.org/90838 | 00:15 |
| fungi | jeblair: mordred: that's why i mentioned jeepyb.gerritdb... we added it a while back but not everything in jeepyb is going through it for db access yet (and that should be considered a bug) | 00:15 |
| mordred | fungi: ++ | 00:15 |
| mordred | jeblair: jenkins broke your other thing too - I | 00:16 |
| mordred | I'm fixing | 00:16 |
| *** SumitNaiksatam has quit IRC | 00:16 | |
| jeblair | mordred: i'm on that one | 00:16 |
| mordred | k | 00:16 |
| mordred | jeblair: btw - thank god the problem isn't actually, you know - a fundamental issue with zuul and gerrit 2.8 | 00:16 |
| jeblair | mordred: kow | 00:16 |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/config: Move configured gerrit replica to /opt/lib/git https://review.openstack.org/90856 | 00:17 |
| mordred | fungi: https://review.openstack.org/#/c/90852/ | 00:17 |
| jeblair | mordred, fungi: btw, i've been keeping https://etherpad.openstack.org/p/J8WNqyP2dC up to date with changes that must merge before we can return to normal | 00:17 |
| mordred | jeblair: excellent | 00:18 |
| jeblair | 2014-04-29 00:18:58,457 DEBUG zuul.Gerrit: Waiting for <Change 0x7fc8c4761dd0 90858,1> to appear in git repo | 00:19 |
| jeblair | 2014-04-29 00:19:03,547 DEBUG zuul.Gerrit: Change <Change 0x7fc8c4761dd0 90858,1> is in the git repo | 00:19 |
| *** gokrokve has quit IRC | 00:19 | |
| jeblair | woohoo ^ | 00:19 |
| *** jeblair changes topic to "Gerrit upgraded to 2.8. See: https://wiki.openstack.org/wiki/GerritUpgrade Known problems with experimental pipelines and formatting of test reports." | 00:19 | |
| *** zehicle_at_dell has joined #openstack-infra | 00:20 | |
| jeblair | merging the remove force-message change | 00:20 |
| openstackgerrit | Monty Taylor proposed a change to openstack-infra/jeepyb: Get the db host from the gerrit config too https://review.openstack.org/90857 | 00:20 |
| mordred | ++ | 00:20 |
| *** sarob has quit IRC | 00:20 | |
| openstackgerrit | A change was merged to openstack-infra/config: Actually disable trivial_rebase_hook https://review.openstack.org/90853 | 00:21 |
| *** gondoi is now known as zz_gondoi | 00:21 | |
| *** prad has quit IRC | 00:22 | |
| openstackgerrit | Monty Taylor proposed a change to openstack-infra/config: Update acls to enforce pushing signed tags https://review.openstack.org/90842 | 00:24 |
| *** bradm has joined #openstack-infra | 00:24 | |
| jeblair | fungi: ^ that lgtm | 00:27 |
| fungi | jeblair: tm too | 00:28 |
| fungi | no longer removing branch creation perms | 00:28 |
| openstackgerrit | A change was merged to openstack-infra/jeepyb: Get the db host from the gerrit config too https://review.openstack.org/90857 | 00:29 |
| *** yidclare has quit IRC | 00:29 | |
| openstackgerrit | A change was merged to openstack-infra/config: Add launchpadlib to review.o.o https://review.openstack.org/90852 | 00:29 |
| *** bhuvan has quit IRC | 00:29 | |
| openstackgerrit | A change was merged to openstack-infra/config: Remove force-message from zuul layout https://review.openstack.org/90832 | 00:29 |
| *** lcostantino has quit IRC | 00:29 | |
| openstackgerrit | A change was merged to openstack-infra/config: Plumb mysql host through gerrit.pp https://review.openstack.org/90837 | 00:29 |
| openstackgerrit | Monty Taylor proposed a change to openstack-infra/config: update docs and All-Project configuration for gerrit 2.8 https://review.openstack.org/70818 | 00:29 |
| mordred | ^^ I just updated zaro's patch - the config section in it now matches what's in production | 00:30 |
| *** nati_uen_ has quit IRC | 00:30 | |
| *** nati_ueno has joined #openstack-infra | 00:31 | |
| *** nati_ueno has quit IRC | 00:31 | |
| jeblair | i ran puppet on zuul; that change is applied now and puppet in clean (except for the global authorized keys errors) | 00:31 |
| fungi | mordred: including the runas addition for bootstrappers? | 00:31 |
| jeblair | i will re-enable puppet on zuul | 00:31 |
| *** mburned is now known as mburned_out | 00:32 | |
| mordred | fungi: oh! no - what does that look like again? (did you apply that to all-projects? | 00:32 |
| *** jreypo has quit IRC | 00:32 | |
| fungi | mordred: (global perm i added so i could impersonate users to set workflow -1 for their wio changes earlier) | 00:32 |
| openstackgerrit | A change was merged to openstack-infra/config: Move configured gerrit replica to /opt/lib/git https://review.openstack.org/90856 | 00:32 |
| mordred | fungi: cool. I'd love to add that | 00:32 |
| fungi | yes, to all-projects... did it through the horrible webui :/ | 00:32 |
| * mordred is going to get All-Projects into our config dir - doesn't really care for the manual nature of it | 00:33 | |
| mordred | same with teh dashboards | 00:33 |
| jeblair | mordred: ++ | 00:33 |
| openstackgerrit | A change was merged to openstack-infra/jeepyb: Use configured mysql host https://review.openstack.org/90838 | 00:33 |
| fungi | mordred: agreed, we could in theory update it through m-p, just can't bootstrap a new gerrit that way obviously | 00:33 |
| jeblair | i believe that's all outstanding changes merged; i'll see if puppet is clean on review | 00:33 |
| jeblair | i thought someone had worked out the minimum perms needed to let m-p bootstrap; so we can document those | 00:35 |
| mordred | jeblair: I thought yolanda did | 00:35 |
| mordred | jeblair: if I wanted to start chasing down the ugly status breakage - any thoughts on where I'd start? | 00:36 |
| fungi | mordred: jeblair: right, though it ended up being juju charm secret sauce for ubuntu cloud, though the concept could probably be repeated | 00:36 |
| mordred | fungi: I'm sure we can ansible it | 00:36 |
| mordred | fungi: or, you know, just write a bash script | 00:37 |
| fungi | yep | 00:37 |
| *** msabramo has joined #openstack-infra | 00:37 | |
| * mordred will say - working with dib at the end of last week made him really appreciate NOT having to learn a meta language | 00:37 | |
| jeblair | fungi, mordred: couple of things missing in puppet. patch in a sec | 00:37 |
| mordred | the learning curve was "what sequence do these thigns run in, which run in the image and which run outside, and what paths are pre-set in env vars" | 00:38 |
| fungi | the basic takeaway is that you need a minimal but syntactically correct nugget of gerrit acl config *and* groups file written to disk with the gerrit service not yet started, enough to allow project-bootstrappers to push the rest through the api | 00:38 |
| openstackgerrit | A change was merged to openstack-infra/config: Update acls to enforce pushing signed tags https://review.openstack.org/90842 | 00:39 |
| mordred | jeblair: since the acl change landed - when you re-run puppet, we may want to follow that with an m-p run | 00:40 |
| *** mrodden has quit IRC | 00:40 | |
| mordred | OR - we could trigger than just by landing the gertty patch once you've re-enabled puppet | 00:40 |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/config: Set github replicationDelay to match (1 sec) https://review.openstack.org/90859 | 00:40 |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/config: Enable run-as in gerrit https://review.openstack.org/90860 | 00:40 |
| jeblair | mordred, fungi: ^ those two are outstanding local-fs vs puppet changes | 00:41 |
| *** mbacchi has quit IRC | 00:41 | |
| *** SumitNaiksatam has joined #openstack-infra | 00:42 | |
| fungi | i could have sworn i added a change identical to https://review.openstack.org/90860 | 00:44 |
| jeblair | fungi: you did https://review.openstack.org/#/c/90814/1 | 00:44 |
| fungi | ahh, i can abandon https://review.openstack.org/90814 | 00:44 |
| openstackgerrit | Monty Taylor proposed a change to openstack-infra/config: update docs and All-Project configuration for gerrit 2.8 https://review.openstack.org/70818 | 00:44 |
| jeblair | sorry i didn't see that | 00:44 |
| *** nati_ueno has joined #openstack-infra | 00:44 | |
| jeblair | i mean, again :) | 00:44 |
| jeblair | i did approve it but it conflicted or something. possibly bogus. | 00:45 |
| jeblair | during the broken times | 00:45 |
| mordred | fungi, jeblair ^^ docs updated to included the runAs capability | 00:45 |
| fungi | brokentimes | 00:45 |
| openstackgerrit | Monty Taylor proposed a change to openstack-infra/config: Add vim to the list of explicit things on servers https://review.openstack.org/90786 | 00:46 |
| *** pcrews has joined #openstack-infra | 00:46 | |
| mordred | neat | 00:46 |
| mordred | I just used "Cherry Pick To" on the same branch to re-parent a patch without downloading it | 00:46 |
| vishy | is recheck still working? https://review.openstack.org/#/c/88416/ | 00:46 |
| jeblair | mordred: i like that you call 'vim' an 'explicit thing' | 00:46 |
| jeblair | vishy: we've been restarting things all day, that was likely lost; should work now | 00:47 |
| mordred | jeblair: :) | 00:48 |
| mordred | fungi: so - as a note from your time away from us ... | 00:48 |
| *** jeblair changes topic to "Gerrit upgraded to 2.8. See: https://wiki.openstack.org/wiki/GerritUpgrade Known problem with formatting of test reports." | 00:48 | |
| mordred | fungi: if you're root on puppetmaster - triggering puppet on one of the nodes is as simple as "ssh $nodename" | 00:48 |
| mordred | fungi: because of the captive command ssh stuff | 00:49 |
| jeblair | fungi: disabling puppet on a node requires running "sudo puppet agent --disable" on the node | 00:49 |
| mordred | and yes - that ^^ | 00:49 |
| mordred | because there are now no daemons | 00:49 |
| jeblair | fungi: (that's in the docs, but i think we just discovered a problem with docs publishing) | 00:50 |
| openstackgerrit | A change was merged to openstack-infra/config: Set github replicationDelay to match (1 sec) https://review.openstack.org/90859 | 00:50 |
| fungi | jeblair: mordred: awesome. i figured out the --disable already since puppet agent obviously wasn't running as a daemon any longer. i should have expected the ssh bit following our discussion about tying shell commands to authorized keys | 00:51 |
| openstackgerrit | A change was merged to openstack-infra/config: Enable run-as in gerrit https://review.openstack.org/90860 | 00:51 |
| jeblair | running puppet noop on review | 00:52 |
| mordred | jeblair: awesome | 00:52 |
| jeblair | okay well that's useless because of the acl change. :) | 00:52 |
| *** etoews has joined #openstack-infra | 00:52 | |
| jeblair | running again and capturing output... | 00:52 |
| jeblair | lgtm, running for real | 00:53 |
| *** ramashri has quit IRC | 00:54 | |
| jeblair | mordred: puppet is running m-p because of the acl changes | 00:54 |
| *** mrodden has joined #openstack-infra | 00:55 | |
| mordred | jeblair: kk | 00:55 |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/config: Remove "Code Review" from Gerrit site header https://review.openstack.org/90862 | 01:03 |
| jeblair | puppet/m-p finished | 01:03 |
| jeblair | running puppet once more | 01:04 |
| jeblair | erm, it seems to have removed melange and melangeclient acl files | 01:04 |
| mordred | hrm | 01:05 |
| mordred | jeblair: so - looking at the test results html stuff - gerrit seems to be stripping class and span from the emittted html | 01:05 |
| *** sweston has quit IRC | 01:06 | |
| jeblair | mordred: any idea about the melange thing? | 01:06 |
| jeblair | because that's triggering another run of manage-projects | 01:06 |
| jeblair | -rw-r--r-- 1 root root 0 Apr 29 01:07 python-melangeclient.config | 01:07 |
| *** marcoemorais has quit IRC | 01:07 | |
| *** jbroome has joined #openstack-infra | 01:08 | |
| jeblair | mordred: could m-p be writing those? | 01:08 |
| mordred | jeblair: yes. yes it could | 01:08 |
| *** jbroome has left #openstack-infra | 01:08 | |
| jeblair | mordred: that seems weird to me; i think of that directory as input files | 01:08 |
| mordred | I do too - I think it may be old logic that we don't care about anymore | 01:09 |
| mordred | jeblair: write_acl_config | 01:09 |
| openstackgerrit | Monty Taylor proposed a change to openstack-infra/jeepyb: Stop writing project configs https://review.openstack.org/90865 | 01:10 |
| mordred | jeblair: ^^ | 01:10 |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/config: Add empty melange and python-melangeclient config files https://review.openstack.org/90866 | 01:11 |
| jeblair | mordred: ^^ | 01:11 |
| openstackgerrit | A change was merged to openstack-dev/pbr: README.rst: tweaks https://review.openstack.org/89277 | 01:12 |
| jeblair | mordred: i don't see melange files on the old server though :/ | 01:12 |
| *** gyee has quit IRC | 01:13 | |
| *** asettle has quit IRC | 01:13 | |
| *** gokrokve has joined #openstack-infra | 01:14 | |
| jeblair | mordred: i don't understand what changed | 01:15 |
| jeblair | i don't really want to enable puppet yet because i think manage-projects will run every time | 01:15 |
| jeblair | mordred: do you think the 0-byte file change is safe? | 01:16 |
| *** gokrokve_ has joined #openstack-infra | 01:17 | |
| jeblair | okay, it's starting to feel like i'm the only one in the room now | 01:18 |
| mordred | jeblair: sorry - was in another window | 01:18 |
| jeblair | i guess we'll leave things as they are | 01:18 |
| jeblair | oh | 01:18 |
| mordred | and my notifications are b0rken | 01:18 |
| mordred | jeblair: I'm not sure what's changed | 01:18 |
| mordred | I don't think the 0-byte change is dangerous | 01:18 |
| *** gokrokve has quit IRC | 01:18 | |
| mordred | but ... why is it trying to make an openstack/melange? | 01:19 |
| mordred | we don't have that in projects.yaml any more do we? | 01:19 |
| mordred | oh - we still do | 01:19 |
| *** sdake_ has quit IRC | 01:19 | |
| jeblair | mordred: we added it back | 01:20 |
| *** gokrokve_ has quit IRC | 01:20 | |
| jeblair | bug 1288485 | 01:20 |
| uvirtbot | Launchpad bug 1288485 in openstack-ci "Old projects listed in ls-projects may not be clonable" [Medium,Fix released] https://launchpad.net/bugs/1288485 | 01:20 |
| mordred | jeblair: oh - you know what? | 01:20 |
| mordred | jeblair: oh - neermind- you rsynced the dirs | 01:21 |
| jeblair | mordred: so if you like the 0 byte files change, can you approve that and we can see if that makes m-p stop running? | 01:21 |
| mordred | yup | 01:22 |
| mordred | jeblair: done | 01:22 |
| *** sweston has joined #openstack-infra | 01:23 | |
| *** asettle-bfast is now known as asettle | 01:23 | |
| *** nati_uen_ has joined #openstack-infra | 01:23 | |
| *** sdake_ has joined #openstack-infra | 01:24 | |
| *** sdake_ has quit IRC | 01:24 | |
| *** sdake_ has joined #openstack-infra | 01:24 | |
| openstackgerrit | A change was merged to openstack-infra/config: Add empty melange and python-melangeclient config files https://review.openstack.org/90866 | 01:26 |
| *** gokrokve has joined #openstack-infra | 01:26 | |
| *** nati_ueno has quit IRC | 01:27 | |
| jeblair | running puppet | 01:27 |
| *** psedlak has quit IRC | 01:28 | |
| mordred | jeblair: I'm going to muck with review-dev to try out a few things for comment links | 01:28 |
| jeblair | mordred: ok. i'm assuming that something about the html changed -- the test report beautification actually matches the transformed html output that gerrit produces for list items | 01:29 |
| jeblair | mordred: that's probably pretty fragile | 01:29 |
| *** gokrokve has quit IRC | 01:30 | |
| jeblair | mordred: fwiw, i figured that one was less urgent and could wait until tomorrow | 01:30 |
| jeblair | mordred: puppet is happy now | 01:30 |
| jeblair | re-enabling on review | 01:30 |
| mordred | jeblair: yeah - I was just looking while you were running puppet | 01:31 |
| jeblair | SergeyLukjanov, mordred, fungi: puppet is running on all the hosts again; it should be safe to approve puppet changes | 01:31 |
| mordred | jeblair: my current hunch is that they're stripping the html of "unsafe" things | 01:31 |
| jeblair | s/running/enabled/ (habits die hard) | 01:32 |
| *** yaguang has joined #openstack-infra | 01:32 | |
| mordred | jeblair: when I open it up in developer tools - the span and class portions of the html are not there | 01:32 |
| mordred | but the li is | 01:32 |
| mordred | oh - I see what you're saying now thought perhsp | 01:32 |
| *** sandywalsh has quit IRC | 01:32 | |
| mordred | ok. | 01:32 |
| jeblair | mordred: well, the li comes from gerrit's own transform... | 01:32 |
| jeblair | ya | 01:32 |
| * mordred long day face | 01:32 | |
| jeblair | mordred: indeed. i think i'm going to sign off now | 01:33 |
| jeblair | i am becoming dangerous around production systems | 01:33 |
| mordred | jeblair: ok. I know what the problem is on the test results | 01:33 |
| jeblair | mordre: okay i can hang for another minute. :) | 01:33 |
| mordred | target="_blank" | 01:33 |
| mordred | has been added | 01:33 |
| mordred | which is not in our regex | 01:34 |
| jeblair | that'll do it | 01:34 |
| mordred | I'll test a patch on review-dev | 01:34 |
| *** gokrokve has joined #openstack-infra | 01:35 | |
| *** psedlak has joined #openstack-infra | 01:35 | |
| jeblair | mordred: ok, i am going to afk for now. thanks for finding that. | 01:35 |
| mordred | jeblair: have a fun night! | 01:36 |
| *** gokrokve has quit IRC | 01:38 | |
| *** etoews has quit IRC | 01:40 | |
| *** gyee has joined #openstack-infra | 01:42 | |
| zaro | mordred: did you find replication docs? | 01:48 |
| *** zehicle has joined #openstack-infra | 01:48 | |
| mordred | zaro: I did not - but it turns out it was a config error on our part | 01:48 |
| mordred | we moved the local replica dir and, well, we forgot to mve it in puppet | 01:48 |
| zaro | it's in the replication plugin, https://gerrit.googlesource.com/plugins/replication/+/stable-2.8/src/main/resources/Documentation | 01:48 |
| *** etoews has joined #openstack-infra | 01:49 | |
| *** adalbas has quit IRC | 01:49 | |
| *** yjiang has joined #openstack-infra | 01:50 | |
| *** zehicle_at_dell has quit IRC | 01:51 | |
| *** zhiyan_ is now known as zhiyan | 01:52 | |
| mordred | ah - resources | 01:54 |
| mordred | thanks zaro | 01:55 |
| *** markmcclain has quit IRC | 02:00 | |
| *** adalbas has joined #openstack-infra | 02:01 | |
| *** nati_uen_ has quit IRC | 02:02 | |
| openstackgerrit | Monty Taylor proposed a change to openstack-infra/config: Update the URL match pattern https://review.openstack.org/90872 | 02:03 |
| mordred | fungi, jeblair, SergeyLukjanov: my brain cannot wrap itself around how to test this ^^ | 02:04 |
| mordred | but I wanted to capture it as the current state of my brain | 02:04 |
| fungi | taking a look | 02:04 |
| fungi | and then i am going to pretend that it's late here and that i need to be productive some tomorrow | 02:04 |
| fungi | mordred: that makes sense actually. i noticed that links in gerrit 2.8 now open in a new target rather than in the current window/tab | 02:05 |
| mordred | fungi: I mean, it's not that destructive - we _could_ just apply it and restart gerrit and see if it fixes it | 02:06 |
| fungi | it's possible to test on review-dev it we want | 02:06 |
| fungi | not so much testing the puppetry, but manually adjusting the gerrit.config in a corresponding manner | 02:07 |
| mordred | fungi: how do we get text in there to test ? | 02:07 |
| mordred | fungi: I've applied it on review-dev already | 02:07 |
| mordred | but I wasn't sure what I should put in the comment | 02:07 |
| mordred | as input | 02:07 |
| fungi | ahh, unordered list elements i thought | 02:07 |
| fungi | checking to see where i found it hard-coded into zuul previously | 02:08 |
| *** smarcet has quit IRC | 02:09 | |
| *** sweston has quit IRC | 02:09 | |
| fungi | mordred: http://git.openstack.org/cgit/openstack-infra/zuul/tree/zuul/scheduler.py#n1486 | 02:10 |
| jeblair | just leave a comment on a change in dev with the text of a comment that zuul has left in prod | 02:11 |
| jeblair | mordred: ^ | 02:11 |
| *** Sukhdev has joined #openstack-infra | 02:12 | |
| fungi | oh, i may be looking at the wrong bit of code there | 02:12 |
| fungi | yeah, the formatReport() method on n1410 | 02:13 |
| Sukhdev | fungi: Can I ask you a question? | 02:14 |
| fungi | Sukhdev: you can try, though i'm fairly brain-dead at this point | 02:14 |
| Sukhdev | fungi: :-) | 02:14 |
| Sukhdev | fungi: My third party test system, when posting vote on Gerrit, it comes back and gives me a warning that remote host identification has changed | 02:15 |
| fungi | Sukhdev: yes, we changed the gerrit service host keys today when gerrit was upgraded, as a safety precaution | 02:16 |
| Sukhdev | fungi: has anything changed on the gerrit server | 02:16 |
| fungi | Sukhdev: have you been following the openstack-dev mailing list? | 02:17 |
| Sukhdev | fungi: Ah ha - that explains it…. | 02:17 |
| Sukhdev | fungi: no, I was caught up with a critical issue at work | 02:17 |
| Sukhdev | fungi: any quick fix that I can use to get this going? | 02:17 |
| *** sweston has joined #openstack-infra | 02:18 | |
| fungi | Sukhdev: http://lists.openstack.org/pipermail/openstack-dev/2014-April/033908.html | 02:18 |
| fungi | Sukhdev: the fix will depend on the configuration of your system (particularly, what software is connecting to the gerrit api endpoint) | 02:20 |
| mordred | jeblair: nope. can't get it to be better :( | 02:20 |
| Sukhdev | fungi: I use the following command to post the vote - ssh -p 29418 arista-test@review.openstack.org gerrit review -m ' Arista third party testing PASSED | 02:21 |
| *** wenlock has joined #openstack-infra | 02:21 | |
| Sukhdev | fungi: so, if I delete the host_id from my .ssh/known_hosts - that should do the trick, right? | 02:22 |
| fungi | Sukhdev: yes, and then accept the new key | 02:22 |
| Sukhdev | fungi: how do I accept the new key? | 02:22 |
| fungi | Sukhdev: you could try running 'ssh -p 29418 arista-test@review.openstack.org' locally as the same user which your ci runs under | 02:23 |
| openstackgerrit | A change was merged to openstack-infra/jenkins-job-builder: Update authorship information https://review.openstack.org/89912 | 02:23 |
| fungi | and then when openssh prompts you about the new key fingerprint, respond "yes" | 02:24 |
| Sukhdev | fungi: cool….let me give it a try | 02:24 |
| *** alexpilotti has quit IRC | 02:32 | |
| *** ramashri has joined #openstack-infra | 02:38 | |
| *** Ryan_Lane has quit IRC | 02:38 | |
| *** terryyao has joined #openstack-infra | 02:39 | |
| Sukhdev | fungi: Thanks - I was able to get the correct keys… | 02:40 |
| *** mriedem has quit IRC | 02:41 | |
| *** ramashri has quit IRC | 02:43 | |
| fungi | Sukhdev: you're welcome | 02:43 |
| *** markwash has quit IRC | 02:44 | |
| *** Sukhdev has quit IRC | 02:47 | |
| terryyao | hello, who can tell me the progress on https://review.openstack.org/90017 | 02:53 |
| *** bknudson has quit IRC | 02:54 | |
| zaro | terryyao: we don't approval until a change gets two +2 | 02:54 |
| openstackgerrit | Dan Prince proposed a change to openstack-infra/config: Add yum mirror support for lightweight mirrors https://review.openstack.org/90874 | 02:54 |
| openstackgerrit | Dan Prince proposed a change to openstack-infra/config: Add yum.openstack.org lightweight Fedora 20 mirror https://review.openstack.org/90875 | 02:54 |
| openstackgerrit | Dan Prince proposed a change to openstack-infra/config: Add yum.openstack.org lightweight Fedora 20 mirror https://review.openstack.org/90875 | 03:00 |
| openstackgerrit | Dan Prince proposed a change to openstack-infra/config: Add yum mirror support for lightweight mirrors https://review.openstack.org/90874 | 03:00 |
| *** yjiang has quit IRC | 03:01 | |
| *** yjiang|2 has joined #openstack-infra | 03:02 | |
| terryyao | can any core reviewer take a look at https://review.openstack.org/90017? | 03:17 |
| *** arnaud__ has quit IRC | 03:18 | |
| *** Longgeek has joined #openstack-infra | 03:22 | |
| *** mengxd has joined #openstack-infra | 03:27 | |
| *** gokrokve has joined #openstack-infra | 03:27 | |
| tjones | mattoliverau: anteaya: just letting you know i tried to push the review again and it worked just fine. | 03:32 |
| *** etoews has quit IRC | 03:32 | |
| mengxd | jeblair: ping | 03:34 |
| *** msabramo has quit IRC | 03:35 | |
| *** msabramo has joined #openstack-infra | 03:36 | |
| mattoliverau | tjones: awesome, I was just out for lunch was going to ping you when I got back :) Glad it's working now! | 03:37 |
| *** praneshp has quit IRC | 03:55 | |
| *** aysyd has quit IRC | 03:57 | |
| *** jamespage has quit IRC | 04:00 | |
| *** gnuoy has quit IRC | 04:00 | |
| jhesketh_ | is gerrit not accessible at the moment? | 04:00 |
| *** jamespage has joined #openstack-infra | 04:00 | |
| *** gnuoy has joined #openstack-infra | 04:00 | |
| *** jamespage has quit IRC | 04:00 | |
| *** jamespage has joined #openstack-infra | 04:00 | |
| openstackgerrit | Marc Abramowitz proposed a change to openstack-infra/jenkins-job-builder: Add more tests https://review.openstack.org/90524 | 04:01 |
| *** dripton has quit IRC | 04:03 | |
| *** gyee has quit IRC | 04:04 | |
| *** marcoemorais has joined #openstack-infra | 04:04 | |
| *** dripton has joined #openstack-infra | 04:04 | |
| openstackgerrit | Joshua Hesketh proposed a change to openstack-infra/nodepool: Make template and node hostnames configurable https://review.openstack.org/90883 | 04:04 |
| *** chandan_kumar has joined #openstack-infra | 04:05 | |
| *** mengxd has quit IRC | 04:05 | |
| *** marcoemorais has quit IRC | 04:08 | |
| *** sweston has quit IRC | 04:12 | |
| *** morganfainberg is now known as morganfainberg_Z | 04:20 | |
| *** praneshp has joined #openstack-infra | 04:21 | |
| *** gokrokve has quit IRC | 04:22 | |
| *** praneshp_ has joined #openstack-infra | 04:22 | |
| *** Ryan_Lane has joined #openstack-infra | 04:23 | |
| *** marcoemorais1 has joined #openstack-infra | 04:24 | |
| *** praneshp has quit IRC | 04:26 | |
| *** praneshp_ is now known as praneshp | 04:26 | |
| *** sweston has joined #openstack-infra | 04:27 | |
| *** marcoemorais1 has quit IRC | 04:29 | |
| *** marcoemorais has joined #openstack-infra | 04:30 | |
| *** ildikov has quit IRC | 04:33 | |
| *** sweston_ has joined #openstack-infra | 04:34 | |
| *** sweston has quit IRC | 04:34 | |
| *** mspreitz has joined #openstack-infra | 04:36 | |
| mspreitz | I just got this complaint: The RSA host key for [review.openstack.org]:29418 has changed | 04:37 |
| mspreitz | Did that key really change? | 04:37 |
| jhesketh_ | mspreitz: yes | 04:44 |
| jhesketh_ | mspreitz: gerrit was upgraded | 04:45 |
| jhesketh_ | mspreitz: https://wiki.openstack.org/wiki/GerritUpgrade#DNS_Spoofing_Warning | 04:45 |
| mspreitz | jhesketh_: thanks | 04:45 |
| jhesketh_ | mspreitz: there was also an email to the -dev list with PGP signature confirming the new host key (if you want more confidence then review.openstack.org confirming it's own key) | 04:45 |
| *** enykeev has joined #openstack-infra | 04:52 | |
| *** ArxCruz has quit IRC | 04:53 | |
| enykeev | Hey! Is gerrit still down? http://cl.ly/VEKQ | 04:54 |
| enykeev | or I'm just missing something | 04:55 |
| *** talluri has joined #openstack-infra | 04:56 | |
| *** harlowja is now known as harlowja_away | 04:56 | |
| mattoliverau | enykeev:gerrit is back up | 04:57 |
| enykeev | then why I'm getting 503? | 04:57 |
| mattoliverau | enykeev: try shift+refresh | 04:57 |
| enykeev | tried cmd+r, no luck | 04:58 |
| enykeev | same with another browser | 04:58 |
| mattoliverau | not cmd+r (on a mac) acutally shift+<refresh button on browser> to refresh from cache, or try clearing your cache and then going to the site. | 05:00 |
| mattoliverau | enykeev: ^^ it is up, and everyone I've talked to has had the same problem. | 05:01 |
| enykeev | k, now it works. Thanks. | 05:01 |
| mattoliverau | enykeev: no problem, enjoy the new gerrit :) | 05:01 |
| *** Sukhdev has joined #openstack-infra | 05:08 | |
| *** gokrokve has joined #openstack-infra | 05:14 | |
| *** yjiang|2 has quit IRC | 05:24 | |
| openstackgerrit | Joshua Hesketh proposed a change to openstack-infra/nodepool: Make template and node hostnames configurable https://review.openstack.org/90883 | 05:34 |
| *** unicell has joined #openstack-infra | 05:34 | |
| *** ildikov has joined #openstack-infra | 05:35 | |
| *** pcrews has quit IRC | 05:43 | |
| *** Ryan_Lane1 has joined #openstack-infra | 05:47 | |
| *** gokrokve_ has joined #openstack-infra | 05:48 | |
| *** Ryan_Lane1 has quit IRC | 05:50 | |
| *** gokrokve has quit IRC | 05:52 | |
| *** sdake_ has quit IRC | 05:54 | |
| *** gokrokve_ has quit IRC | 05:57 | |
| *** chandan_kumar has quit IRC | 06:01 | |
| *** ashishsnigam has quit IRC | 06:06 | |
| *** ashishsnigam has joined #openstack-infra | 06:06 | |
| *** etoews has joined #openstack-infra | 06:09 | |
| *** praneshp has quit IRC | 06:12 | |
| *** etoews has quit IRC | 06:13 | |
| *** chandan_kumar has joined #openstack-infra | 06:16 | |
| *** xchu has joined #openstack-infra | 06:18 | |
| *** jamielennox is now known as jamielennox|away | 06:18 | |
| *** chandan_kumar has quit IRC | 06:22 | |
| *** mspreitz has quit IRC | 06:27 | |
| *** gokrokve has joined #openstack-infra | 06:27 | |
| *** mrmartin has joined #openstack-infra | 06:28 | |
| *** viktors|afk is now known as viktors | 06:29 | |
| *** gokrokve_ has joined #openstack-infra | 06:29 | |
| *** gokrokve has quit IRC | 06:32 | |
| *** gokrokve_ has quit IRC | 06:34 | |
| *** sweston_ has quit IRC | 06:38 | |
| *** ArxCruz has joined #openstack-infra | 06:38 | |
| *** flaper87|afk is now known as flaper87 | 06:39 | |
| *** chandan_kumar has joined #openstack-infra | 06:40 | |
| *** ArxCruz has quit IRC | 06:40 | |
| *** e0ne has joined #openstack-infra | 06:42 | |
| *** che-arne has quit IRC | 06:51 | |
| *** d0ugal has joined #openstack-infra | 06:52 | |
| *** unicell has quit IRC | 06:54 | |
| *** gokrokve has joined #openstack-infra | 06:57 | |
| *** afazekas_ has joined #openstack-infra | 06:58 | |
| *** _nadya_ has joined #openstack-infra | 06:58 | |
| *** thomasbiege has joined #openstack-infra | 07:03 | |
| *** mrmartin has quit IRC | 07:03 | |
| *** jlibosva has joined #openstack-infra | 07:05 | |
| *** gema has joined #openstack-infra | 07:05 | |
| *** jcoufal has joined #openstack-infra | 07:06 | |
| *** Sukhdev has quit IRC | 07:08 | |
| *** _nadya_ has quit IRC | 07:08 | |
| *** thomasbiege has quit IRC | 07:08 | |
| *** sdake has quit IRC | 07:10 | |
| *** wenlock has quit IRC | 07:11 | |
| *** mrmartin has joined #openstack-infra | 07:11 | |
| *** _nadya_ has joined #openstack-infra | 07:13 | |
| *** ihrachyshka has joined #openstack-infra | 07:14 | |
| mattoliverau | I'm calling it a day, have a great night/day all! | 07:14 |
| *** zns has quit IRC | 07:16 | |
| *** thomasbiege has joined #openstack-infra | 07:20 | |
| *** jgallard has joined #openstack-infra | 07:20 | |
| *** maxbit has joined #openstack-infra | 07:27 | |
| *** reed has joined #openstack-infra | 07:27 | |
| *** sweston has joined #openstack-infra | 07:34 | |
| *** sweston has quit IRC | 07:39 | |
| *** thomasbiege has quit IRC | 07:41 | |
| *** gema has quit IRC | 07:43 | |
| *** hashar has joined #openstack-infra | 07:43 | |
| *** Ryan_Lane has quit IRC | 07:49 | |
| *** dizquierdo has joined #openstack-infra | 07:50 | |
| *** jpich has joined #openstack-infra | 07:54 | |
| *** rcarrill` has quit IRC | 07:54 | |
| *** skolekonov has joined #openstack-infra | 07:54 | |
| *** rcarrillocruz has joined #openstack-infra | 07:54 | |
| *** sdake has joined #openstack-infra | 08:01 | |
| *** ashishsnigam has quit IRC | 08:04 | |
| openstackgerrit | Antoine Musso proposed a change to openstack-infra/zuul: Pass pngcrush on status png files https://review.openstack.org/89865 | 08:04 |
| *** ildikov has quit IRC | 08:05 | |
| openstackgerrit | Steven Hardy proposed a change to openstack-infra/reviewstats: Add thomasspatzier to heat-core https://review.openstack.org/90942 | 08:08 |
| *** etoews has joined #openstack-infra | 08:11 | |
| *** yassine has joined #openstack-infra | 08:12 | |
| openstackgerrit | Antoine Musso proposed a change to stackforge/python-jenkins: Wrap tests and linting with tox https://review.openstack.org/89521 | 08:15 |
| *** bauzas has joined #openstack-infra | 08:15 | |
| *** etoews has quit IRC | 08:16 | |
| *** skolekonov has left #openstack-infra | 08:17 | |
| *** derekh has joined #openstack-infra | 08:17 | |
| *** amcrn has quit IRC | 08:18 | |
| SergeyLukjanov | morning folks | 08:18 |
| *** gema has joined #openstack-infra | 08:19 | |
| derekh | Does the need to be approved again https://review.openstack.org/#/c/90757/1 ? I'm guessing it didn't merge because of gerrit upgrade | 08:19 |
| openstackgerrit | A change was merged to openstack-infra/jenkins-job-builder: Add some more Trove classifiers https://review.openstack.org/90674 | 08:20 |
| derekh | SergeyLukjanov: thanks | 08:20 |
| SergeyLukjanov | derekh, np | 08:20 |
| tchaypo | dstufft: "write a simple test", he said | 08:23 |
| tchaypo | If I'd know at the time that this was going to lead to me copying a function called find-links2 to make a new one called find-links3 that's one character different, I might have felt less cheerful.. | 08:25 |
| *** ildikov has joined #openstack-infra | 08:26 | |
| *** darraghb has joined #openstack-infra | 08:27 | |
| openstackgerrit | A change was merged to openstack-infra/config: Run check-tripleo jobs on stale changes https://review.openstack.org/85854 | 08:31 |
| openstackgerrit | Fabien Boucher proposed a change to openstack-infra/config: Fix typo in ACL. https://review.openstack.org/90945 | 08:33 |
| *** sweston has joined #openstack-infra | 08:34 | |
| rcarrillocruz | fungi: thanks for the approval on https://review.openstack.org/#/c/86382/ , just saw it... | 08:37 |
| *** ihrachyshka has quit IRC | 08:37 | |
| *** jooools has joined #openstack-infra | 08:38 | |
| *** mattymo|home has joined #openstack-infra | 08:39 | |
| *** sweston has quit IRC | 08:39 | |
| *** ihrachyshka has joined #openstack-infra | 08:39 | |
| *** rcarrill` has joined #openstack-infra | 08:40 | |
| *** rcarrillocruz has quit IRC | 08:42 | |
| openstackgerrit | A change was merged to openstack-infra/jenkins-job-builder: add tests for shell builder https://review.openstack.org/90480 | 08:43 |
| openstackgerrit | Sergey Lukjanov proposed a change to openstack-infra/config: Update the URL match pattern https://review.openstack.org/90872 | 08:44 |
| openstackgerrit | A change was merged to openstack-infra/config: Fix typo in ACL. https://review.openstack.org/90945 | 08:44 |
| openstackgerrit | A change was merged to openstack-infra/config: Remove "Code Review" from Gerrit site header https://review.openstack.org/90862 | 08:46 |
| *** mrmartin has quit IRC | 08:48 | |
| *** mrmartin has joined #openstack-infra | 08:49 | |
| openstackgerrit | A change was merged to openstack-infra/config: update docs and All-Project configuration for gerrit 2.8 https://review.openstack.org/70818 | 08:52 |
| *** _nadya_ has quit IRC | 08:52 | |
| * SergeyLukjanov trying to use new screen | 08:54 | |
| SergeyLukjanov | it's not too bad as it can be, but.... | 08:54 |
| *** _nadya_ has joined #openstack-infra | 08:56 | |
| openstackgerrit | A change was merged to openstack-infra/config: Update the URL match pattern https://review.openstack.org/90872 | 08:58 |
| *** Mithrandir has quit IRC | 09:01 | |
| *** adalbas has quit IRC | 09:01 | |
| *** derekh has quit IRC | 09:03 | |
| *** gema has quit IRC | 09:06 | |
| SergeyLukjanov | fungi, should it be approved again https://review.openstack.org/#/c/81622/? | 09:09 |
| SergeyLukjanov | jeblair, mordred ^^ | 09:09 |
| *** thomasbiege has joined #openstack-infra | 09:10 | |
| mrmartin | Hi SergeyLukjanov, git review -s doesn't work for me: http://paste.openstack.org/show/77640/ had you met with something similar? | 09:10 |
| SergeyLukjanov | fungi, jeblair, mordred, don't worry, it's just new screen (/me don't see the merged label) | 09:11 |
| *** che-arne has joined #openstack-infra | 09:11 | |
| SergeyLukjanov | mrmartin, have you already updated host key? | 09:11 |
| SergeyLukjanov | https://wiki.openstack.org/wiki/GerritUpgrade#DNS_Spoofing_Warning | 09:11 |
| mrmartin | not yet | 09:12 |
| *** Mithrandir has joined #openstack-infra | 09:13 | |
| *** thomasbiege1 has joined #openstack-infra | 09:13 | |
| *** etoews has joined #openstack-infra | 09:13 | |
| mrmartin | SergeyLukjanov, thanks works for me now! | 09:14 |
| *** chandan_kumar has quit IRC | 09:14 | |
| *** chandan_kumar has joined #openstack-infra | 09:15 | |
| *** Mithrandir has quit IRC | 09:16 | |
| *** thomasbiege has quit IRC | 09:17 | |
| *** etoews has quit IRC | 09:18 | |
| *** bauzas has left #openstack-infra | 09:18 | |
| *** dangers is now known as dangers_away | 09:19 | |
| *** Mithrandir has joined #openstack-infra | 09:20 | |
| *** markmc has joined #openstack-infra | 09:20 | |
| *** rcarrillocruz has joined #openstack-infra | 09:21 | |
| *** Milind has joined #openstack-infra | 09:21 | |
| openstackgerrit | A change was merged to openstack-infra/config: Check for new files as well as changed files https://review.openstack.org/90843 | 09:22 |
| *** rcarrill` has quit IRC | 09:22 | |
| *** jp_at_hp has joined #openstack-infra | 09:23 | |
| *** ominakov has joined #openstack-infra | 09:23 | |
| openstackgerrit | A change was merged to openstack-infra/config: limit the width of vote columns https://review.openstack.org/90844 | 09:24 |
| openstackgerrit | A change was merged to openstack-infra/config: Add vim to the list of explicit things on servers https://review.openstack.org/90786 | 09:24 |
| Milind | tempest.thirdparty.boto.test_ec2_instance_run.InstanceRunTest.test_run_stop_terminate_instance_with_tags[gate,smoke] ... FAIL | 09:25 |
| Milind | is this a known bug ? | 09:25 |
| *** thomasbiege1 has quit IRC | 09:26 | |
| *** bauzas has joined #openstack-infra | 09:26 | |
| *** WaltNik has joined #openstack-infra | 09:30 | |
| ihrachyshka | any plans to revert to a nice gate jobs results view with green and red colouring? | 09:36 |
| ihrachyshka | (re gerrit) | 09:37 |
| StevenK | Is anyone getting 503s from gerrit on https? | 09:40 |
| StevenK | Anyone else, that is ... :-( | 09:40 |
| *** ociuhandu has quit IRC | 09:41 | |
| BobBallAway | I'm getting them yes | 09:43 |
| fifieldt | any OpenStack IRC ops around? | 09:44 |
| bauzas | StevenK: I'm having no troubles with https | 09:45 |
| fifieldt | jeblair, mordred, jeremyb - had a spammer in #openstack, probably needs a ban | 09:45 |
| *** terryyao has quit IRC | 09:47 | |
| *** marcoemorais has quit IRC | 09:49 | |
| StevenK | bauzas: That's odd. IPv4 or v6? | 09:53 |
| bauzas | IPv4 | 09:53 |
| bauzas | StevenK: IPv4 sorry | 09:53 |
| bauzas | StevenK: and the cert is identified | 09:54 |
| * StevenK drops his IPv6 default route to see if that works | 09:54 | |
| bauzas | (btw. this major upgrade is great, thanks mordred, fungi , jeblair and other missing) | 09:54 |
| StevenK | Right, IPv4 works, IPv6 gives 503 | 09:56 |
| *** gema has joined #openstack-infra | 10:01 | |
| *** ociuhandu has joined #openstack-infra | 10:02 | |
| *** andreaf has joined #openstack-infra | 10:06 | |
| *** aglarendil has joined #openstack-infra | 10:08 | |
| *** BobBallAway is now known as BobBall | 10:09 | |
| BobBall | ohhhh | 10:09 |
| *** mrmartin has quit IRC | 10:09 | |
| *** alexpilotti has joined #openstack-infra | 10:09 | |
| *** mdenny has quit IRC | 10:17 | |
| *** Milind has quit IRC | 10:18 | |
| *** yfried has joined #openstack-infra | 10:19 | |
| Adri2000 | hi | 10:19 |
| Adri2000 | at https://review.openstack.org/#/c/89534/ why did jenkins decide it would recheck (and do nothing afterwards)? | 10:19 |
| Adri2000 | is it a consequence of the gerrit upgrade? | 10:20 |
| yfried | jesusaurus: ping | 10:20 |
| *** _nadya_ has quit IRC | 10:25 | |
| *** e0ne has quit IRC | 10:26 | |
| *** e0ne has joined #openstack-infra | 10:27 | |
| yfried | can anyone here help me with logstash? | 10:27 |
| *** e0ne has quit IRC | 10:32 | |
| *** lucasagomes has joined #openstack-infra | 10:32 | |
| lucasagomes | hi there, I can't push any changes to gerrit after the update, I'm getting a "fatal: Could not read from remote repository." | 10:32 |
| lucasagomes | anyone with the same problem? | 10:32 |
| *** mrmartin has joined #openstack-infra | 10:32 | |
| *** _nadya_ has joined #openstack-infra | 10:34 | |
| lucasagomes | already fixed following https://wiki.openstack.org/wiki/GerritUpgrade | 10:36 |
| *** jgallard has quit IRC | 10:38 | |
| openstackgerrit | Sean Dague proposed a change to openstack-infra/config: gerrit css tweaks to make it more like old gerrit https://review.openstack.org/90967 | 10:46 |
| openstackgerrit | Nikola Dipanov proposed a change to openstack-dev/hacking: Add an import whitelist helper factory https://review.openstack.org/80018 | 10:57 |
| openstackgerrit | Nikola Dipanov proposed a change to openstack-dev/hacking: Add TestCase assertion methods for testing checks https://review.openstack.org/80278 | 10:57 |
| *** dizquierdo has quit IRC | 10:58 | |
| *** e0ne has joined #openstack-infra | 11:07 | |
| *** e0ne has quit IRC | 11:09 | |
| *** e0ne has joined #openstack-infra | 11:09 | |
| *** e0ne has quit IRC | 11:10 | |
| *** e0ne has joined #openstack-infra | 11:11 | |
| *** gema has quit IRC | 11:11 | |
| *** lucasagomes has left #openstack-infra | 11:11 | |
| openstackgerrit | Nikola Dipanov proposed a change to openstack-dev/hacking: Add an import whitelist helper factory https://review.openstack.org/80018 | 11:12 |
| *** enikanorov__ is now known as enikanorov_ | 11:13 | |
| *** mrmartin has quit IRC | 11:15 | |
| *** e0ne has quit IRC | 11:16 | |
| *** fifieldt_ has joined #openstack-infra | 11:16 | |
| YorikSar | Adri2000: it looks like your change got into check queue just before upgrade. | 11:17 |
| YorikSar | Adri2000: You can recheck it manually. | 11:17 |
| *** fifieldt has quit IRC | 11:17 | |
| openstackgerrit | Sean Dague proposed a change to openstack-infra/config: gerrit css tweaks to make it more like old gerrit https://review.openstack.org/90967 | 11:28 |
| *** e0ne has joined #openstack-infra | 11:29 | |
| *** aglarendil has left #openstack-infra | 11:29 | |
| *** yaguang has quit IRC | 11:30 | |
| *** mburned_out is now known as mburned | 11:32 | |
| *** _nadya_ has quit IRC | 11:32 | |
| *** julim has joined #openstack-infra | 11:34 | |
| *** rlandy has joined #openstack-infra | 11:40 | |
| SergeyLukjanov | fifieldt, is op still needed? | 11:41 |
| openstackgerrit | Nikita Konovalov proposed a change to openstack-infra/storyboard: Teams API https://review.openstack.org/90984 | 11:43 |
| *** dprince has joined #openstack-infra | 11:47 | |
| *** sandywalsh has joined #openstack-infra | 11:48 | |
| SergeyLukjanov | jeblair, mordred, fungi, IIRC we need to restart gerrit for https://review.openstack.org/#/c/90872/, am I right? | 11:49 |
| *** che-arne has quit IRC | 11:50 | |
| openstackgerrit | Nikita Konovalov proposed a change to openstack-infra/storyboard: Teams API https://review.openstack.org/90984 | 11:50 |
| openstackgerrit | Dan Prince proposed a change to openstack-infra/config: Add yum.openstack.org lightweight Fedora 20 mirror https://review.openstack.org/90875 | 11:51 |
| openstackgerrit | Dan Prince proposed a change to openstack-infra/config: Add yum mirror support for lightweight mirrors https://review.openstack.org/90874 | 11:51 |
| *** ildikov has quit IRC | 11:54 | |
| *** mrmartin has joined #openstack-infra | 11:54 | |
| *** pdmars has joined #openstack-infra | 11:56 | |
| *** pdmars has quit IRC | 11:57 | |
| *** pdmars_ has joined #openstack-infra | 11:57 | |
| bookwar | hi, i have a problem with our third-party Jenkins, it seems Gerrit Trigger gets all the changes but can't leave a comment. Has something changed in the way service accounts work? Mine is called "fuel-ci" | 11:58 |
| *** pdmars_ has quit IRC | 11:58 | |
| bookwar | for this patchset here https://review.openstack.org/#/c/90957/ i have "INFO: Notifying BuildStarted to gerrit: gerrit approve 90957,1 --message 'Build Started http://fuel-jenkins.mirantis.com/job/stackforge-verify-fuel-web/3297/ (2/2)' --verified 0 --code-review 0" in Jenkins logs, but it is nothing in comments | 12:01 |
| *** pdmars has joined #openstack-infra | 12:01 | |
| *** sdake has quit IRC | 12:01 | |
| *** prad has joined #openstack-infra | 12:01 | |
| *** sdake has joined #openstack-infra | 12:01 | |
| *** sdake has quit IRC | 12:01 | |
| *** sdake has joined #openstack-infra | 12:01 | |
| openstackgerrit | Sean Dague proposed a change to openstack-infra/config: gerrit css tweaks to make it more like old gerrit https://review.openstack.org/90967 | 12:02 |
| ihrachyshka | is gerrit-to-launchpad auto-update broken? | 12:05 |
| openstackgerrit | Dan Prince proposed a change to openstack-infra/config: Add yum.openstack.org lightweight Fedora 20 mirror https://review.openstack.org/90875 | 12:05 |
| openstackgerrit | Dan Prince proposed a change to openstack-infra/config: Add yum mirror support for lightweight mirrors https://review.openstack.org/90874 | 12:05 |
| *** afazekas_ has quit IRC | 12:05 | |
| *** dkranz has joined #openstack-infra | 12:12 | |
| bookwar | ok, found it myself, as 'gerrit approve' is deprecated, we need to update our Gerrit Triiger plugin to work with new gerrit properly | 12:14 |
| *** etoews has joined #openstack-infra | 12:16 | |
| *** afazekas_ has joined #openstack-infra | 12:19 | |
| *** prad has quit IRC | 12:19 | |
| *** weshay has joined #openstack-infra | 12:20 | |
| *** etoews has quit IRC | 12:20 | |
| *** gema has joined #openstack-infra | 12:22 | |
| *** rfolco has joined #openstack-infra | 12:23 | |
| *** zhiyan is now known as zhiyan_ | 12:23 | |
| *** gema has left #openstack-infra | 12:25 | |
| *** e0ne has quit IRC | 12:26 | |
| *** e0ne has joined #openstack-infra | 12:27 | |
| *** etoews has joined #openstack-infra | 12:29 | |
| *** aysyd has joined #openstack-infra | 12:30 | |
| *** weshay has quit IRC | 12:31 | |
| *** jgallard has joined #openstack-infra | 12:32 | |
| *** dims has quit IRC | 12:33 | |
| *** lcostantino has joined #openstack-infra | 12:37 | |
| *** miqui has joined #openstack-infra | 12:39 | |
| *** rcarrillocruz has quit IRC | 12:41 | |
| *** alexpilotti has quit IRC | 12:46 | |
| *** rcarrillocruz has joined #openstack-infra | 12:47 | |
| *** dims has joined #openstack-infra | 12:48 | |
| *** rcarrill` has joined #openstack-infra | 12:51 | |
| *** rcarrillocruz has quit IRC | 12:53 | |
| *** dkranz has quit IRC | 12:53 | |
| *** rcarrillocruz has joined #openstack-infra | 12:54 | |
| *** adalbas has joined #openstack-infra | 12:56 | |
| *** rcarrill` has quit IRC | 12:56 | |
| *** ildikov has joined #openstack-infra | 12:56 | |
| *** eharney has quit IRC | 12:58 | |
| *** heyongli has joined #openstack-infra | 13:01 | |
| fungi | SergeyLukjanov: you were able to work out the commentlink fix (per 90872) i take it? | 13:02 |
| SergeyLukjanov | fungi, morning | 13:02 |
| SergeyLukjanov | fungi, I've check it on a bunch of html and it's working ok | 13:02 |
| *** fifieldt_ is now known as fifieldt | 13:03 | |
| fungi | oh, i see. mordred had target before href when gerrit was printing it after instead | 13:03 |
| SergeyLukjanov | fungi, exactly | 13:03 |
| fungi | i was too bleary-eyed to see straight by that point last night and missed it entirely | 13:03 |
| SergeyLukjanov | fungi, I've updated the CR and approved it | 13:04 |
| SergeyLukjanov | fungi, and only after it merged I remember that we need to restart gerrit to apply it | 13:04 |
| fungi | great! but yeah, it takes a gerrit restart to apply configuration changes, and since this is a cosmetic fix we may want to wait a little and batch it up with other fixes we may need to restart for anyway | 13:04 |
| SergeyLukjanov | fungi, sure | 13:05 |
| BobBall | Any chance you could workflow+1 https://review.openstack.org/#/c/90660/ again fungi? You did so yesterday but then I guess something got restarted, so the job was lost | 13:06 |
| SergeyLukjanov | fungi, new screen is a bit hmmmm | 13:06 |
| *** doug-fish has joined #openstack-infra | 13:07 | |
| sdague | BobBall: I can trigger it | 13:09 |
| BobBall | thanks sdague. | 13:09 |
| fungi | ahh, yep, i see what went sideways there | 13:09 |
| sdague | though it doesn't seem like it's taking the additional approve | 13:09 |
| sdague | fungi: I guess we are missing a trigger? | 13:10 |
| sdague | BobBall: so actually, because we were within clean check, you actually just needed a recheck no bug to get jenkins to pay attention to it | 13:11 |
| fungi | i added a +1 workflow to all the previously approved changes after the upgrade, adding myself to the project bootstrappers group so i'd have the necessary permissions, but when i removed myself from that group again afterward it also made my vote disappear since i no longer had permission | 13:11 |
| fungi | then adding myself back to that group, it appears again | 13:11 |
| fungi | the way gerrit applies acls is very, very bizarre | 13:11 |
| BobBall | oh OK | 13:11 |
| *** smarcet has joined #openstack-infra | 13:11 | |
| BobBall | oh, another question, were you aware that some ppl (such as me on one machine) can't access gerrit but others can? | 13:12 |
| fungi | BobBall: have they restarted their browser sessions? | 13:12 |
| BobBall | I get a service temporarily unavailable | 13:12 |
| *** yfried has quit IRC | 13:12 | |
| BobBall | I will testthat | 13:13 |
| BobBall | d'oh | 13:13 |
| BobBall | sorry | 13:13 |
| BobBall | I hadn't :) | 13:13 |
| fungi | i've noticed that some browsers have a tendency to not re-resolve dns unless you take more extreme measures | 13:13 |
| sdague | heh | 13:13 |
| *** weshay has joined #openstack-infra | 13:13 | |
| fungi | firefox does that to me, and it's especially annoying when i migrate a device in and out of networks with split horizon dns | 13:14 |
| *** che-arne has joined #openstack-infra | 13:16 | |
| *** mriedem has joined #openstack-infra | 13:19 | |
| *** zz_gondoi is now known as gondoi | 13:19 | |
| *** signed8bit has joined #openstack-infra | 13:20 | |
| *** flaper87 is now known as flaper87|afk | 13:20 | |
| *** e0ne has quit IRC | 13:21 | |
| *** e0ne has joined #openstack-infra | 13:21 | |
| *** Ryan_Lane has joined #openstack-infra | 13:22 | |
| *** mwagner_lap has quit IRC | 13:22 | |
| openstackgerrit | Sergey Lukjanov proposed a change to openstack-infra/config: Add slukjanov to users https://review.openstack.org/91012 | 13:23 |
| sdague | fungi: any idea if anyone's working on adding the formatted comments back in? | 13:24 |
| *** jgrimm has joined #openstack-infra | 13:24 | |
| *** bknudson has joined #openstack-infra | 13:24 | |
| sdague | for gerrit | 13:24 |
| fungi | sdague: for the zuul comments? | 13:24 |
| sdague | yeh | 13:24 |
| fungi | sdague: see SergeyLukjanov's recent scrollback | 13:24 |
| fungi | the fix is already merged, we just need a gerrit restart | 13:24 |
| SergeyLukjanov | sdague, I've fixed mordred's patch and approve it already, it waits for gerrit restart and we'd like to combine it with something more important | 13:24 |
| sdague | ok, cool | 13:25 |
| *** flaper87|afk is now known as flaper87 | 13:25 | |
| sdague | reading tempest test returns without that is ... challenging - https://review.openstack.org/#/c/81872/10 | 13:25 |
| SergeyLukjanov | fungi, I've decided to add my public key :) | 13:25 |
| fungi | on the assumption we'll find something more important to restart it for anyway. it is only the morning after, after all | 13:25 |
| sdague | fungi: sure | 13:25 |
| sdague | fungi: I've got another css change as well - https://review.openstack.org/#/c/90967/ | 13:26 |
| *** Ryan_Lane has quit IRC | 13:26 | |
| fungi | sdague: saw it. that probably also needs a restart since, as you observed, gerrit compiles all the css | 13:26 |
| sdague | yep | 13:26 |
| sdague | well, GerritSite.css isn't actually compiled in | 13:26 |
| sdague | it's just inlined | 13:27 |
| fungi | ahh | 13:27 |
| SergeyLukjanov | fungi, do you have 5 mins to remove a bunch of branches for sahara repos? | 13:28 |
| fungi | SergeyLukjanov: how many is a bunch? but sure, it's quick | 13:28 |
| *** jcoufal has quit IRC | 13:28 | |
| *** alexpilotti has joined #openstack-infra | 13:28 | |
| SergeyLukjanov | fungi, I'd like to remove all stable/0.* from all repos | 13:29 |
| SergeyLukjanov | fungi, all sahara repos | 13:29 |
| fungi | looking | 13:29 |
| SergeyLukjanov | fungi, so, it's about 7-8 branches | 13:29 |
| *** jcoufal has joined #openstack-infra | 13:29 | |
| SergeyLukjanov | fungi, thank you! | 13:29 |
| SergeyLukjanov | fungi, it's about only openstack/*sahara* repos | 13:31 |
| *** mengxd has joined #openstack-infra | 13:31 | |
| *** mfer has joined #openstack-infra | 13:33 | |
| *** alexpilotti has quit IRC | 13:34 | |
| mengxd | Does anyone know the core member list for the openstack-infrastructure project? | 13:35 |
| fungi | SergeyLukjanov: but you want to leave the stable/icehouse and milestone-proposed branches? | 13:35 |
| pabelanger | mengxd, https://review.openstack.org/#/admin/groups/306,members should get you started | 13:36 |
| SergeyLukjanov | fungi, yup, please | 13:36 |
| *** mnaser has joined #openstack-infra | 13:36 | |
| openstackgerrit | Sergey Lukjanov proposed a change to openstack-infra/config: Open test results comment in new tab/window https://review.openstack.org/91013 | 13:37 |
| openstackgerrit | Ilya Sviridov proposed a change to openstack-infra/config: Added gate-magnetodb-devstack-dsvm as gate https://review.openstack.org/91014 | 13:37 |
| *** matty_dubs|gone is now known as matty_dubs | 13:37 | |
| fungi | SergeyLukjanov: so just deleting these... http://paste.openstack.org/show/77706/ | 13:38 |
| SergeyLukjanov | fungi, yup | 13:40 |
| *** markmcclain has joined #openstack-infra | 13:40 | |
| *** mengxd has quit IRC | 13:41 | |
| fungi | SergeyLukjanov: done. double-check that i didn't screw anything up | 13:41 |
| fungi | mengxd: if you want to bring something up in the openstack infra team meeting later today, you may want to get it on the agenda ahead of time... https://wiki.openstack.org/wiki/Meetings/InfraTeamMeeting#Agenda_for_next_meeting | 13:42 |
| anteaya | fungi: I can read backscroll or ask, so I'll ask first, is there anything useful I can do to help you/someone else? | 13:42 |
| SergeyLukjanov | fungi, everything is correct, thank you! | 13:42 |
| fungi | anteaya: so far not much seems to have come up. however i'm 2+ weeks behind on e-mail so digging myself out may unearth some pending tasks | 13:43 |
| *** mbacchi has joined #openstack-infra | 13:43 | |
| anteaya | fungi: okay will start to read backscroll, let me know if there is something you unearth | 13:43 |
| *** markmcclain has quit IRC | 13:45 | |
| *** homeless has joined #openstack-infra | 13:47 | |
| openstackgerrit | Marc Abramowitz proposed a change to openstack-infra/jenkins-job-builder: Add more tests https://review.openstack.org/90524 | 13:53 |
| openstackgerrit | Nikita Konovalov proposed a change to openstack-infra/storyboard: Project Groups API https://review.openstack.org/90736 | 13:53 |
| *** flaper87 is now known as flaper87|afk | 13:53 | |
| openstackgerrit | A change was merged to openstack-infra/storyboard-webclient: Favicon Whitelabel https://review.openstack.org/90167 | 13:55 |
| openstackgerrit | A change was merged to openstack-infra/storyboard-webclient: Removed logo images https://review.openstack.org/90169 | 13:56 |
| *** flaper87|afk is now known as flaper87 | 13:56 | |
| *** prad has joined #openstack-infra | 13:58 | |
| *** mrodden has quit IRC | 13:59 | |
| openstackgerrit | A change was merged to openstack-infra/storyboard-webclient: Updated alignment on detail page. https://review.openstack.org/90506 | 13:59 |
| *** pcrews has joined #openstack-infra | 14:00 | |
| *** heyongli has quit IRC | 14:00 | |
| *** dkranz has joined #openstack-infra | 14:01 | |
| openstackgerrit | A change was merged to openstack-infra/storyboard: Users can be filtered with LIKE clauses https://review.openstack.org/90217 | 14:02 |
| openstackgerrit | A change was merged to openstack-infra/storyboard: Projects can be filtered with LIKE clauses https://review.openstack.org/90475 | 14:03 |
| *** derekh has joined #openstack-infra | 14:03 | |
| openstackgerrit | Nikita Konovalov proposed a change to openstack-infra/storyboard: Added test for migrations https://review.openstack.org/90047 | 14:03 |
| openstackgerrit | A change was merged to openstack-infra/storyboard-webclient: Users and projects may now be selected using typeahead search https://review.openstack.org/90502 | 14:09 |
| *** thomasem has joined #openstack-infra | 14:09 | |
| openstackgerrit | Nikita Konovalov proposed a change to openstack-infra/storyboard: Project Groups API https://review.openstack.org/90736 | 14:09 |
| *** yfried has joined #openstack-infra | 14:11 | |
| *** mrodden has joined #openstack-infra | 14:12 | |
| *** pblaho has joined #openstack-infra | 14:13 | |
| *** msabramo has quit IRC | 14:16 | |
| *** ashishsnigam has joined #openstack-infra | 14:20 | |
| mordred | fungi: think we should restart gerrit to pick up SergeyLukjanov's fix? | 14:22 |
| *** david-lyle_ has joined #openstack-infra | 14:22 | |
| *** zhiyan_ is now known as zhiyan | 14:23 | |
| SergeyLukjanov | mordred, we think that we should wait for some more important stuff | 14:24 |
| fungi | mordred: yes, though just wanted to make sure if other issues turn up which also need a restart, that we lump them in rather than having more than one restart today (if possible) | 14:24 |
| mordred | kk | 14:24 |
| *** mdenny has joined #openstack-infra | 14:24 | |
| jeblair | good morning all | 14:25 |
| fungi | mordred: sdague also had a css patch which i believe requires a restart as well | 14:25 |
| SergeyLukjanov | jeblair, morning | 14:25 |
| jeblair | https://review.openstack.org/#/c/90967/ | 14:25 |
| jeblair | that's sdague's css fix which i just +2d (thanks) | 14:25 |
| *** markmcclain has joined #openstack-infra | 14:25 | |
| *** david-lyle_ is now known as david-lyle | 14:25 | |
| jeblair | (i hesitated a moment about the nonvotable background thing -- but i think with our mix of code review and workflow acls, it's actually not useful information for us) | 14:26 |
| *** blamar has joined #openstack-infra | 14:26 | |
| fungi | agreed (and approved) | 14:26 |
| SergeyLukjanov | and I approved https://review.openstack.org/#/c/90862/ to remove "Code Review" from header | 14:27 |
| jeblair | (it would pretty much tell you that most people can't vote verified, and some other people are _either_ the owner or core) | 14:27 |
| SergeyLukjanov | jeblair, yup, it's not very useful IMO | 14:27 |
| *** changbl has quit IRC | 14:27 | |
| jeblair | zaro: is there a way to remove the "Monitoring" menu entry? i don't think we'll ever use it (we can always directly navigate to it), and it takes up a lot of space | 14:28 |
| *** dizquierdo has joined #openstack-infra | 14:28 | |
| *** thedodd has joined #openstack-infra | 14:29 | |
| jeblair | (it's... unfortunate... that javamelody monitoring gets a top level menu item and project dashboards can't get one at all anywhere) | 14:29 |
| * SergeyLukjanov don't see monitoring menu item :( | 14:30 | |
| jeblair | SergeyLukjanov: we have it configured for admin-only. we should probably consider making you a gerrit admin. :) | 14:30 |
| SergeyLukjanov | jeblair, oh, got it :) | 14:30 |
| openstackgerrit | A change was merged to openstack-infra/config: gerrit css tweaks to make it more like old gerrit https://review.openstack.org/90967 | 14:31 |
| gabriel-bezerra | Hi, I've got 2 +1s on this review, but Jenkins didn't give the check or verify. How can I fix this? https://review.openstack.org/#/c/90771/ | 14:31 |
| *** mwagner_lap has joined #openstack-infra | 14:31 | |
| jeblair | gabriel-bezerra: "recheck no bug" | 14:31 |
| rcarrillocruz | recheck no bug i guess | 14:31 |
| jeblair | gabriel-bezerra: we restarted zuul a lot yesterday for the gerrit upgrade, the jobs were probably lost during that | 14:31 |
| *** ArxCruz has joined #openstack-infra | 14:32 | |
| gabriel-bezerra | oh, good.. I was looking for some dumb bug created just to start new rechecks.. | 14:32 |
| SergeyLukjanov | SergeyLukjanov, gerrit admin could be probably useful - I can help with stuff like removing branches, adding to groups, etc. // if it's in admin permissions :) | 14:33 |
| SergeyLukjanov | heh, it's not for myself, jeblair :_ | 14:33 |
| gabriel-bezerra | thank you very much | 14:33 |
| openstackgerrit | Ilya Sviridov proposed a change to openstack-infra/config: Added new python-magnetodbclient project to Stackforge https://review.openstack.org/91050 | 14:33 |
| *** msabramo has joined #openstack-infra | 14:33 | |
| *** wenlock has joined #openstack-infra | 14:34 | |
| mordred | jeblair: yeah - I really wish we could re-work the top menu | 14:34 |
| sdague | ok, it's taking me a while to sort out my custom queries again in the new language. I'll do an updated blog post later with new recipes | 14:35 |
| mordred | sdague: if you get some things sorted out - we could potentially add them as custom dashboards | 14:36 |
| anteaya | right now our known problem is formatting of test reports, what is the issue with test report formatting? | 14:36 |
| sdague | mordred: sure | 14:36 |
| sdague | is there a way for me to register such things? | 14:36 |
| mordred | anteaya: SergeyLukjanov landed a patch to fix it | 14:36 |
| anteaya | great | 14:36 |
| mordred | sdague: not yet - at the moment it takesa core | 14:36 |
| anteaya | well done SergeyLukjanov | 14:36 |
| sdague | I've basically got a number of queries that I use to take slices of things | 14:36 |
| mordred | sdague: but I'd very much like to make them files in git | 14:37 |
| sdague | because my total reviewable list is too long to manage | 14:37 |
| *** e0ne has quit IRC | 14:37 | |
| sdague | mordred: ok, just let me know how to contribute them | 14:37 |
| *** bnemec has quit IRC | 14:39 | |
| anteaya | SergeyLukjanov: is this the patch in question? https://review.openstack.org/#/c/91013/1 | 14:39 |
| mordred | sdague: it's really easy to make them - all you need is the gerrit queries- this is the important changes dashboard: http://paste.openstack.org/show/77720/ | 14:39 |
| mordred | sdague: which is here: https://review.openstack.org/#/projects/All-Projects,dashboards/important-changes:important-changes-dashboard if you haven't foudn it | 14:39 |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/config: Add Gertty project https://review.openstack.org/89457 | 14:41 |
| jeblair | my new projects have sat so long their configuration is out of date :( | 14:41 |
| *** andreaf has quit IRC | 14:41 | |
| jeblair | SergeyLukjanov: you may want to look at that change and check out the ACL changes between the last two patchsets | 14:42 |
| *** pblaho has quit IRC | 14:42 | |
| jeblair | SergeyLukjanov: that's what those items should look like for all projects now | 14:42 |
| openstackgerrit | Ilya Sviridov proposed a change to openstack-infra/config: Added gate-magnetodb-devstack-dsvm as gate https://review.openstack.org/91014 | 14:42 |
| sdague | mordred: is there a way to apply that to something other than All-Projects? | 14:42 |
| *** bnemec has joined #openstack-infra | 14:42 | |
| jeblair | sdague: works for any project | 14:42 |
| sdague | what's the url syntax to apply to a specific project | 14:42 |
| sdague | or a list | 14:42 |
| *** markmcclain has quit IRC | 14:43 | |
| jeblair | sdague: afaik what project it's attached to just determines when/where it shows up in the ui | 14:43 |
| SergeyLukjanov | anteaya, https://review.openstack.org/#/c/90872/ fix for formatting | 14:43 |
| jeblair | sdague: i think if you wanted to filter by project you'd just put it in the query | 14:43 |
| SergeyLukjanov | anteaya, and https://review.openstack.org/#/c/91013/1 is just a possible improvement | 14:43 |
| sdague | for instance - https://review.openstack.org/#/projects/openstack-dev/grenade,dashboards/important-changes:important-changes-dashboard | 14:43 |
| sdague | doesn't seem to do what I'd think | 14:43 |
| SergeyLukjanov | jeblair, /me looking | 14:44 |
| anteaya | SergeyLukjanov: ah, was looking for it under your name, thanks for the url | 14:44 |
| SergeyLukjanov | jeblair, I'll check all other open new-project requests | 14:44 |
| jeblair | sdague: yeah, that's just grenade inheriting the ability to display the important changes dashboard because it is a child of all-projects | 14:44 |
| *** markmcclain has joined #openstack-infra | 14:44 | |
| sdague | jeblair: right, but doesn't filter based on the project | 14:44 |
| jeblair | sdague: if grenade had a grenade-specific dashboard, it would be available at https://review.openstack.org/#/projects/openstack-dev/grenade,dashboards/important-changes:foo | 14:45 |
| jeblair | sdague: but would not be available at https://review.openstack.org/#/projects/openstack-dev/somethingnotgrenade,dashboards/important-changes:foo | 14:45 |
| jeblair | sdague: yeah, and afaik (but i haven't really looked) there's no way to use $current_project_url_context as a search variable | 14:45 |
| *** viktors is now known as viktors|afk | 14:45 | |
| anteaya | what are the chances I can specify in my preferences that comments are collapse all for me? | 14:45 |
| anteaya | as a personal default | 14:46 |
| sdague | ok, let me etherpad up a couple of things that I think would be useful, and we'll see if they are doable | 14:46 |
| *** zns has joined #openstack-infra | 14:46 | |
| anteaya | it is already there, I found it | 14:46 |
| *** ihrachyshka has quit IRC | 14:46 | |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/config: Create infra-manual project https://review.openstack.org/88099 | 14:47 |
| jeblair | sdague: https://review.openstack.org/Documentation/user-dashboards.html#_token_code_project_code | 14:48 |
| jeblair | sdague: i was completely wrong you can totally use the current project context in a dashboard query. :) | 14:48 |
| sdague | nice | 14:48 |
| sdague | that will make this much better :) | 14:49 |
| jeblair | yeah | 14:49 |
| fungi | while i was out, mriedem requested membership in openstack-ci-bugs on lp (presumably to assist with triage). any objections? should i ask again at the meeting, or just approve? | 14:49 |
| anteaya | SergeyLukjanov: so does 91013 open in a pop-up window? | 14:49 |
| jeblair | fungi: just approve; i don't really know why that's not open | 14:49 |
| fungi | jeblair: approved | 14:50 |
| openstackgerrit | Andreas Jaeger proposed a change to openstack/requirements: Update openstack-doc-tools to 0.13 https://review.openstack.org/90746 | 14:50 |
| mriedem | fungi: jeblair: muwahahaha | 14:52 |
| phschwartz | Morning infra | 14:52 |
| SergeyLukjanov | anteaya, yup | 14:52 |
| anteaya | SergeyLukjanov: but only if you click on it, right? | 14:52 |
| mordred | jeblair: oh! | 14:52 |
| mordred | It is possible to define a default dashboard for a project in the projects project.config file in the refs/meta/config branch: | 14:53 |
| openstackgerrit | A change was merged to openstack-infra/config: Fix minor non-important error in cache_devstack https://review.openstack.org/90775 | 14:53 |
| anteaya | SergeyLukjanov: is there an option to have it not open in a new window? | 14:53 |
| jeblair | mordred: what does 'default' mean? | 14:53 |
| *** bhuvan has joined #openstack-infra | 14:53 | |
| jeblair | phschwartz: good morning | 14:53 |
| mordred | jeblair: I'm not 100% sure - but it certainly seems like something that might be useful given how undiscoverable these things are | 14:54 |
| SergeyLukjanov | anteaya, not add _blank target :) | 14:54 |
| fungi | i personally find that behavior very annoying | 14:55 |
| *** ildikov_ has joined #openstack-infra | 14:55 | |
| anteaya | SergeyLukjanov: sorry I don't understand | 14:56 |
| SergeyLukjanov | fungi, opening in new tab? | 14:56 |
| fungi | browsers implement ways for users to control whether links open in the same place or a new window/tab/whatever. having the application decide that for you is silly | 14:56 |
| fungi | and disruptive | 14:56 |
| SergeyLukjanov | anteaya, I mean that if we're adding target=_blank than link will be opened in new tab on click | 14:56 |
| SergeyLukjanov | fungi, it sounds fair | 14:56 |
| *** ildikov has quit IRC | 14:58 | |
| anteaya | okay, yeah that was what I was trying to get at, and fungi summarized my sense of it nicely | 14:58 |
| SergeyLukjanov | fungi, anteaya, heh, that's funny, but I agree with it and don't know why I've proposed this change :) | 14:59 |
| anteaya | my browser needs to have the last word on how something is opened for me | 14:59 |
| * SergeyLukjanov going to abandon | 14:59 | |
| anteaya | SergeyLukjanov: I've had those moments | 14:59 |
| jeblair | SergeyLukjanov: what change? | 14:59 |
| *** ildikov_ has quit IRC | 14:59 | |
| jeblair | anteaya: see https://review.openstack.org/#/settings/preferences for comment visibility | 15:00 |
| *** derekh has quit IRC | 15:00 | |
| anteaya | jeblair: https://review.openstack.org/#/c/91013/1/modules/openstack_project/manifests/gerrit.pp | 15:00 |
| anteaya | jeblair: yes, found it after I asked about it, thanks | 15:01 |
| *** Longgeek_ has joined #openstack-infra | 15:01 | |
| anteaya | one of these days I will look _before_ I ask | 15:01 |
| mordred | bah | 15:01 |
| mordred | where's teh fun in that | 15:01 |
| *** zns has quit IRC | 15:02 | |
| mordred | jeblair: so - puppet seems to thnk it needs to add our ssh keys every time | 15:02 |
| mordred | jeblair: http://puppetdb.openstack.org/report/jenkins07.openstack.org/5cf45a24153e74f58f94c2fbf7ae389747e6cb56 | 15:03 |
| anteaya | enjoying a nice dinner with jeblair and a few others, which you pay for (thanks monty), that's the fun | 15:03 |
| *** Longgeek has quit IRC | 15:03 | |
| mordred | jeblair: I'll look in to it a little more, but if I can't figure it out, I'll be proposing we revert the use of ssh_authorized_key there | 15:03 |
| *** rcarrillocruz has quit IRC | 15:03 | |
| SergeyLukjanov | mordred, yeah, I've already seen it when approved this change, but I was thinking that it's ok | 15:03 |
| dkranz | fungi: My review bookmarks broke it seems due to (at least) from the query "CodeReview" changing to "Code-Review". Is there some advice on fixing these? | 15:04 |
| SergeyLukjanov | mordred, jeblair, btw it's about https://review.openstack.org/#/c/87340/ | 15:04 |
| *** e0ne has joined #openstack-infra | 15:05 | |
| *** zns has joined #openstack-infra | 15:05 | |
| *** zns has quit IRC | 15:06 | |
| msabramo | Any pbr folks? Could use feedback on https://review.openstack.org/89205 | 15:06 |
| sdague | dkranz: I'm working on that one | 15:06 |
| *** zns has joined #openstack-infra | 15:06 | |
| dkranz | sdague: Thanks! I created them from your so useful email about it. | 15:06 |
| sdague | dkranz: https://review.openstack.org/Documentation/user-search.html is the new docs if you don't want to wait | 15:07 |
| dkranz | sdague: cook thanks | 15:07 |
| sdague | though, based on the previous conversation, we might be able to turn some of these into custom dashboards | 15:07 |
| sdague | so they would be baked into gerrit | 15:07 |
| sdague | which would be super nice | 15:07 |
| *** bauzas has quit IRC | 15:07 | |
| *** bauzas has joined #openstack-infra | 15:07 | |
| mordred | oh. zomg | 15:08 |
| mordred | WOW | 15:08 |
| mordred | puppet is useless | 15:08 |
| mordred | or at least puppetboard is | 15:08 |
| mordred | (actually, I think it's puppet's fault) | 15:08 |
| *** rcarrillocruz has joined #openstack-infra | 15:08 | |
| mordred | err: /Stage[main]/Openstack_project::Users/User::Virtual::Localuser[clarkb]/Ssh_authorized_key[clarkb_keys]: Could not evaluate: Puppet::Util::FileType::FileTypeFlat could not write /home/clarkb/.ssh/authorized_keys: Permission denied - /home/clarkb/.ssh/authorized_keys | 15:08 |
| mordred | that is the error it's not showing us | 15:08 |
| mordred | because, you know, why bother showing an error in the dashboard | 15:09 |
| anteaya | which node? | 15:09 |
| mordred | all of them | 15:09 |
| anteaya | interesting | 15:09 |
| * mordred hitting with angry gorilla power | 15:09 | |
| anteaya | that must be a site | 15:10 |
| *** UtahDave has joined #openstack-infra | 15:10 | |
| anteaya | sight even | 15:10 |
| SergeyLukjanov | fungi, thanks for the comment on https://review.openstack.org/#/c/91012/ - I understand that it doesn't add my acc to any servers, just would like to push my public key in | 15:12 |
| openstackgerrit | Marc Abramowitz proposed a change to openstack-infra/jenkins-job-builder: Fix broken ant example https://review.openstack.org/91061 | 15:12 |
| SergeyLukjanov | fungi, I've commented re agreement with guideline | 15:12 |
| SergeyLukjanov | mordred, oh... I've checked puppetboard after this changes was applied and thought that everything is ok... | 15:14 |
| *** ashishsnigam has quit IRC | 15:14 | |
| openstackgerrit | A change was merged to openstack-infra/config: Add Gertty project https://review.openstack.org/89457 | 15:15 |
| sdague | interesting bit - https://review.openstack.org/#/c/44714/ | 15:15 |
| sdague | merge conflicts no longer trigger a V-2 | 15:16 |
| *** ihrachyshka has joined #openstack-infra | 15:16 | |
| *** gokrokve_ has joined #openstack-infra | 15:18 | |
| openstackgerrit | Graham Hayes proposed a change to openstack-infra/config: Add stackforge/designate-specs https://review.openstack.org/87540 | 15:18 |
| anteaya | sdague: :/ | 15:20 |
| anteaya | it triggers a V+2 | 15:21 |
| *** gokrokve has quit IRC | 15:22 | |
| BobBall | What was gerrit's old IP address? | 15:23 |
| *** Ryan_Lane has joined #openstack-infra | 15:23 | |
| anteaya | BobBall: second line on https://etherpad.openstack.org/p/J8WNqyP2dC | 15:24 |
| BobBall | thanks anteaya | 15:24 |
| anteaya | np | 15:24 |
| BobBall | that was the magic etherpad I was looking for! | 15:24 |
| openstackgerrit | A change was merged to openstack-infra/config: Create infra-manual project https://review.openstack.org/88099 | 15:24 |
| openstackgerrit | Monty Taylor proposed a change to openstack-infra/config: Fix ssh key error and also remove duplicates https://review.openstack.org/91066 | 15:25 |
| anteaya | yay | 15:25 |
| mordred | jeblair, SergeyLukjanov: ^^ that patch itself isn't tested - but the concepts in it have checked out - I'm going to do a quick test run of it on a host in a mordred env | 15:25 |
| *** ildikov has joined #openstack-infra | 15:27 | |
| *** Ryan_Lane has quit IRC | 15:27 | |
| *** ihrachyshka has quit IRC | 15:28 | |
| jeblair | oh new projects merged! | 15:28 |
| anteaya | yay | 15:30 |
| kiall | jeblair: got a minute to rekick that failed tag job I mentioned last night for me? | 15:30 |
| jeblair | mordred: i don't understand "${title}_keyfile" | 15:30 |
| *** ihrachyshka has joined #openstack-infra | 15:30 | |
| *** zhiyan is now known as zhiyan_ | 15:31 | |
| jeblair | kiall: i'm not certain things are working yet -- i pushed a tag yesterday and it didn't show up; we should wait until i get to the bottom of that | 15:32 |
| *** zehicle has quit IRC | 15:32 | |
| kiall | jeblair: No problem, I'll ping you again at some point tomorrow then? | 15:32 |
| jeblair | kiall: sounds good | 15:32 |
| kiall | thanks | 15:32 |
| openstackgerrit | Monty Taylor proposed a change to openstack-infra/config: Fix ssh key error and also remove duplicates https://review.openstack.org/91066 | 15:33 |
| mordred | jeblair: ok- ^^ that one works | 15:33 |
| mordred | jeblair: so - ${title}_keyfile is there to ensure that the file is more 600 and not mode 400 which is what the files our there are | 15:34 |
| jeblair | mordred: okay now "${title}_keyfile" makes sense. :) | 15:34 |
| mordred | ah | 15:34 |
| mordred | :) | 15:34 |
| jeblair | mordred: the lack of name was throwing me off in the old patchset | 15:34 |
| mordred | yah | 15:34 |
| mordred | it was -- not great :) | 15:34 |
| openstackgerrit | Marc Abramowitz proposed a change to openstack-infra/jenkins-job-builder: Fix broken builders-from example https://review.openstack.org/91079 | 15:34 |
| jeblair | i can imagine for once puppet and i were confused by the same thing | 15:35 |
| mordred | this should get us back down to one key each and also give us a way to delete old keys when we cycle them | 15:35 |
| mordred | jeblair: given an inifinte number of ... | 15:35 |
| *** dizquierdo has quit IRC | 15:35 | |
| jeblair | mordred: (we could delete old keys before, we just removed them from the big string) | 15:35 |
| mordred | yup | 15:35 |
| mordred | I mean, this gets us feature parity to the old code | 15:36 |
| mordred | with extra compexity | 15:36 |
| jeblair | mordred: how do we have more than one key each? | 15:36 |
| *** talluri has quit IRC | 15:36 | |
| mordred | we revert the patch that introduced this and go back to the old code | 15:36 |
| anteaya | mordred: did you just make the key id's up? | 15:37 |
| mordred | anteaya: I did not - those were the existing key ids that were originally there | 15:37 |
| *** talluri has joined #openstack-infra | 15:37 | |
| jeblair | mordred: so rolling keys is a little awkward, but i guess we can live with it | 15:37 |
| anteaya | really? | 15:37 |
| anteaya | oh okay, in which case I revoke my -1 | 15:37 |
| anteaya | wonder why russellb would give himself only one L in his name | 15:39 |
| mordred | jeblair: honestly - I find everything about puppet's primitives to manager users and keys to be so ridiculous I think I've given up having an opinion | 15:39 |
| *** msabramo has quit IRC | 15:39 | |
| jeblair | http://logs.openstack.org/e2/e25132cfe539ff3d393882822a66f9c17d9e1f72/release/gerritbot-tarball/2c451be/console.html | 15:40 |
| mordred | that our virtualuser class exists at all should be seen as a giant glaring deficiency | 15:40 |
| jeblair | here's the gerritbot tag error | 15:40 |
| jeblair | mordred: yeah | 15:40 |
| mordred | jeblair: so - the tag didn't replicate? | 15:40 |
| jeblair | yeah, checking zm02's logs now | 15:40 |
| *** talluri has quit IRC | 15:41 | |
| *** zhiyan_ is now known as zhiyan | 15:42 | |
| *** dangers_away is now known as dangers | 15:45 | |
| *** afazekas_ has quit IRC | 15:47 | |
| jeblair | mordred: hrm, i'm not getting it. zm02 updated the repo before that job ran, it should have updated from gerrit directly over ssh, and that git fetch command works for me now. | 15:48 |
| *** ominakov has quit IRC | 15:48 | |
| jeblair | mordred: the local git repo on zm02 has the tag, and zuul hasn't touched it since. | 15:48 |
| *** Longgeek_ has quit IRC | 15:49 | |
| *** Longgeek has joined #openstack-infra | 15:50 | |
| *** sweston has joined #openstack-infra | 15:50 | |
| *** ihrachyshka has quit IRC | 15:51 | |
| *** msabramo has joined #openstack-infra | 15:52 | |
| *** jcoufal has quit IRC | 15:52 | |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/config: Add a release job for sandbox https://review.openstack.org/91084 | 15:52 |
| mordred | jeblair: that makes me less happy | 15:52 |
| jeblair | mordred, fungi, SergeyLukjanov: can we go ahead and merge that so i can test pushing lots of tags? ^ | 15:52 |
| mordred | jeblair: yah | 15:53 |
| fungi | jeblair: lgtm, though you might want it in pre-release too for later testing of regexes? | 15:54 |
| jeblair | sure 1 sec | 15:54 |
| *** matty_dubs is now known as matty_dubs|lunch | 15:54 | |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/config: Add a release job for sandbox https://review.openstack.org/91084 | 15:54 |
| *** julim has quit IRC | 15:55 | |
| *** andreaf has joined #openstack-infra | 15:55 | |
| *** WaltNik has quit IRC | 15:58 | |
| *** bauzas has left #openstack-infra | 15:59 | |
| *** msabramo has quit IRC | 16:02 | |
| Alex_Gaynor | Only the devstack machines are managed by nodepool right? | 16:02 |
| *** gokrokve_ has quit IRC | 16:02 | |
| fungi | Alex_Gaynor: most of our jenkins slaves | 16:03 |
| openstackgerrit | A change was merged to openstack-infra/gear: Add access control https://review.openstack.org/84873 | 16:03 |
| Alex_Gaynor | fungi: does that include the centos ones? | 16:03 |
| fungi | Alex_Gaynor: yes. the bare-centos6 slaves are nodepool-managed | 16:03 |
| fungi | Alex_Gaynor: http://git.openstack.org/cgit/openstack-infra/config/tree/modules/openstack_project/templates/nodepool/nodepool.yaml.erb#n21 | 16:04 |
| Alex_Gaynor | fungi: hmm, there seems to be way fewer centos nodes right now then are needed (like every build is blocked on the python26 builder) | 16:04 |
| fungi | Alex_Gaynor: thanks, i'll see if we're having trouble adding new centos6 nodes for some reason | 16:04 |
| Alex_Gaynor | thanks | 16:04 |
| *** Longgeek has quit IRC | 16:04 | |
| *** ihrachyshka has joined #openstack-infra | 16:04 | |
| *** markmcclain has quit IRC | 16:05 | |
| jeblair | fungi: mordred made a quick script in /var/log/nodepool to count success/failure | 16:06 |
| SlickNik | hey guys. | 16:07 |
| fungi | Alex_Gaynor: looks like we're having centos6 trouble in hpcloud-region-b (about 70 building and 70 deleting, which accounts for the majority of the centos6 nodes at the moment). i'll dig deeper | 16:07 |
| *** msabramo has joined #openstack-infra | 16:07 | |
| *** yassine has quit IRC | 16:07 | |
| anteaya | SlickNik: hey there | 16:07 |
| Alex_Gaynor | fungi: ah, glad I said something! | 16:08 |
| *** derekh has joined #openstack-infra | 16:08 | |
| fungi | Alex_Gaynor: me too. you're our early-warning system ;) | 16:08 |
| SlickNik | The trove "reddwarf" user hasn't been able to post on trove gerrit reviews since the upgrade. It's using the gerrit review plugin. Are there any known issues with this? | 16:08 |
| *** alexpilotti has joined #openstack-infra | 16:09 | |
| *** e0ne has quit IRC | 16:09 | |
| fungi | SlickNik: check your local logs, but you're probably getting stuck on http://lists.openstack.org/pipermail/openstack-dev/2014-April/033908.html | 16:09 |
| anteaya | SlickNik: have you updated the host keys for the reddwarf user: https://wiki.openstack.org/wiki/GerritUpgrade#DNS_Spoofing_Warning | 16:09 |
| anteaya | like fungi said | 16:10 |
| SlickNik | anteaya / fungi: Nope I haven't done that yet, so that's probably what it is. | 16:10 |
| SlickNik | Will try that and see if it fixes it. Thanks! :) | 16:10 |
| *** doug-fish has quit IRC | 16:11 | |
| *** zhiyan is now known as zhiyan_ | 16:11 | |
| mordred | fungi: you may also want to check out the script just so that you know it's there | 16:11 |
| fungi | jeblair: mordred: great--thanks! | 16:12 |
| mordred | http://paste.openstack.org/show/77747/ | 16:12 |
| mordred | there's processing yesterday | 16:12 |
| mordred | btw- lookie there- hpcloud 1.1 is doing well! | 16:12 |
| mordred | and WOW - rax is doing very very very badly | 16:12 |
| mordred | is that related to yesterday's resolv.conf issue? | 16:13 |
| *** jgallard has quit IRC | 16:13 | |
| fungi | mordred: yes, almost certainly | 16:13 |
| mordred | yup. rax numbers in the current log are back to normal | 16:14 |
| mordred | although dfw is very unhappy | 16:14 |
| mordred | ,1100,681,rax-dfw | 16:14 |
| *** jlibosva has quit IRC | 16:14 | |
| *** doug-fish has joined #openstack-infra | 16:14 | |
| mordred | that's 1100 succeded, 681 failed since the log rolled over at 00:00 UTC | 16:14 |
| fungi | yeah | 16:14 |
| phschwartz | mordred: Is that 681 builds failed? | 16:14 |
| mordred | that's 681 node creations failed | 16:15 |
| mordred | phschwartz: so, 681 we tried to spin up a node and something in the process failed before we put it into the pool as an available slave | 16:15 |
| phschwartz | mordred: Let me see if I can get one of our cloud ops to take a look. That is a disturbing number | 16:15 |
| fungi | i'm going to see if i can coax the script into breakdown by provider+label since i have a feeling most/all bare-centos6 nodes in hpcloud-region-b are failing to build | 16:15 |
| phschwartz | mordred: Are there stored logs as to what went wrong? Ie was it an node creation/build issue or configuration inside of the node | 16:16 |
| *** signed8bit has quit IRC | 16:16 | |
| mordred | phschwartz: we test that tons of things work before we consider it "good" - dns has been a recent beast we've been battling | 16:16 |
| mordred | phschwartz: could be either in that count | 16:16 |
| fungi | mordred: one possibility is that one or more of the images failed to update in rax-dfw and still has the chattr +i /etc/resolv.conf going on | 16:16 |
| mordred | phschwartz: so from rax perspective, those could look like "success" | 16:16 |
| mordred | fungi: ++ | 16:16 |
| fungi | check and see if all labels in rax-dfw have corresponding images <24 hours old | 16:17 |
| mordred | phschwartz: we _do_ have logs, I don't think we've audited them fully to make sure they're safe to publish to a location you could get them from | 16:17 |
| phschwartz | mordred: I will err on the worst case and see if I can have a cloud ops person take a look. | 16:17 |
| mordred | fungi: oldest in rax-dfw is 19 hours | 16:18 |
| mordred | fungi: hpcloud-region-b nodes are mostly over 200 hours old | 16:18 |
| mordred | fungi: actually ALL of hp images are - 1.0 regions are in the 400 hours range | 16:19 |
| fungi | oh, right, i deleted them all out of rax yesterday since they were broken, so these failures in rax-dfw very well may not be our doing | 16:19 |
| *** alexpilotti has quit IRC | 16:19 | |
| *** chandan_kumar has quit IRC | 16:19 | |
| fungi | hplcoud 1.0 images mostly don't succeed because of the nova metadata server timeouts, last i saw | 16:20 |
| fungi | can't remember if region b has the same issue | 16:20 |
| *** markmcclain has joined #openstack-infra | 16:20 | |
| phschwartz | mordred, fungi: I have asked our admins to look at our logs and see if they see anything funky in our regions that might cause an issue. As to the image failures, let me know if it looks like there is an actual snapshotting issue with glance for us and I will get some people involved to see what the issue might be. | 16:23 |
| mordred | phschwartz: cool, thanks! | 16:23 |
| mordred | phschwartz: we'd trying to figure out good ways we can provide data to you guys on an ongoing basis | 16:23 |
| mordred | I mean, given that we use a ton of resources quite actively - any way in which we can also be a data collection vehcile is awesome | 16:24 |
| *** yidclare has joined #openstack-infra | 16:24 | |
| *** dangers is now known as dangers_away | 16:24 | |
| phschwartz | mordred: That would be a huge help. Let me know anything you need to help facilitate that, even if I have to work in my spare time to write a log submission api + datastore for you guys to hit. | 16:24 |
| *** BadCub has joined #openstack-infra | 16:24 | |
| *** sweston has quit IRC | 16:24 | |
| jeblair | ssh root@elasticsearch06.openstack.org timeout -s 9 30m puppet agent --onetime --ignorecache --no-daemonize --no-usecacheonfailure --no-splay | 16:25 |
| jeblair | Permission denied (publickey). | 16:25 |
| jeblair | mordred: ^ | 16:25 |
| *** talluri has joined #openstack-infra | 16:25 | |
| mordred | jeblair: the key on es06 looks right | 16:26 |
| jeblair | ssh root@jenkins-dev.openstack.org timeout -s 9 30m puppet agent --onetime --ignorecache --no-daemonize --no-usecacheonfailure --no-splay | 16:27 |
| jeblair | Permission denied (publickey). | 16:27 |
| mordred | jeblair: where are you seeing that? | 16:27 |
| mordred | jeblair: I believe jenkins-dev puppet is still broken and we expect jenkins-dev to not work | 16:28 |
| jeblair | puppet_run_all.log | 16:29 |
| *** salv-orlando_ has joined #openstack-infra | 16:29 | |
| jeblair | mordred: i'm looking into why it's failing on git01 and saw those other errors | 16:29 |
| *** mriedem has quit IRC | 16:29 | |
| mordred | jeblair: but I've just tested both that command line _and_ just ssh elasticsearch06.openstack.org from ci-puppetmaster and both worked | 16:29 |
| derekh | Hi, is anybody willing to pull the trigger on devstack-f20 nodepool instances ? https://review.openstack.org/#/c/86842/ I'll be gone shortly but can be online later to help with any problems building the templates | 16:29 |
| *** salv-orlando has left #openstack-infra | 16:30 | |
| *** salv-orlando has quit IRC | 16:30 | |
| *** salv-orlando_ is now known as salv-orlando | 16:30 | |
| openstackgerrit | A change was merged to openstack-infra/config: Add a release job for sandbox https://review.openstack.org/91084 | 16:30 |
| jeblair | err: Failed to apply catalog: Parameter key failed on Ssh_authorized_key[gerrit-replication-2014-04-25]: Key must not contain whitespace: AAA.... | 16:30 |
| jeblair | mordred: that's why git01 is failing | 16:30 |
| fungi | mordred: the centos6 build failures in hpcloud-region-b look like ssh timeouts after the build completes | 16:30 |
| *** mriedem has joined #openstack-infra | 16:30 | |
| jeblair | fungi, mordred: do we need to bump up our ssh timeouts again? | 16:30 |
| *** marcoemorais has joined #openstack-infra | 16:31 | |
| *** GheRivero has quit IRC | 16:31 | |
| fungi | well, we're also failing to build centos6 images in rackspace at all, so i'm going to start diagnosing that as well | 16:31 |
| mordred | jeblair: I'm very confused - there is no space in the key ... | 16:31 |
| jeblair | mordred: i agree | 16:32 |
| tjones | folks - on this patch turbo hipster gave a -1 and then a +1 but the stats still show -1 https://review.openstack.org/#/c/86443/ can someone take a look? | 16:33 |
| zaro | morning | 16:33 |
| dims | tjones, i am hoping a "recheck migrations" will help. but yes, need to figure out why the -1 is sticky | 16:34 |
| tjones | yeah | 16:34 |
| fungi | tjones: that's very, very odd... | 16:34 |
| tjones | the 2nd migration was ok so i am surprised it did not update the -1 | 16:34 |
| mordred | jeblair: also, the key is on git01.o.o | 16:34 |
| *** ildikov has quit IRC | 16:35 | |
| mordred | jeblair: I wonder - do we need to put quotes around $git_gerrit_ssh_key ? | 16:35 |
| *** Ajaeger has joined #openstack-infra | 16:36 | |
| *** matty_dubs|lunch is now known as matty_dubs | 16:36 | |
| Ajaeger | Hi jeblair, I see on https://review.openstack.org/#/c/82176 "Patch Set 1: (1 inline comment)" - but don't see a comment from you. Any idea what happened? | 16:37 |
| jeblair | Ajaeger: i don't recall leaving an inline comment :/ | 16:37 |
| fungi | i think the centos6 image build failures look like they're because curl isn't following a redirect from https://raw.github.com/pypa/pip/master/contrib/get-pip.py to https://raw.githubusercontent.com/pypa/pip/master/contrib/get-pip.py | 16:37 |
| *** pcrews_ has joined #openstack-infra | 16:38 | |
| jeblair | fungi: i thought we updated that? | 16:38 |
| *** pcrews has quit IRC | 16:38 | |
| fungi | checking | 16:38 |
| mordred | fungi: yes. there is a patch at least in flight to fix that | 16:38 |
| Ajaeger | jeblair: I don't see you voting either ;) This is strange | 16:39 |
| openstackgerrit | Monty Taylor proposed a change to openstack-infra/config: Remove accumulated python cruft https://review.openstack.org/90573 | 16:39 |
| fungi | mordred: fixing http://git.openstack.org/cgit/openstack-infra/config/tree/modules/openstack_project/files/nodepool/scripts/fix_pip.sh#n26 or elsewhere? | 16:39 |
| mordred | fungi: ^^ that's not the one that came in yesterday, but also contains the fix | 16:39 |
| openstackgerrit | Zane Bitter proposed a change to openstack-infra/reviewstats: Add Thomas Spatzier to heat-core https://review.openstack.org/91094 | 16:39 |
| fungi | oh, great | 16:40 |
| mordred | jeblair: have you been able to use REST api against our gerrit? I tried last night (granted I was tired) and got bumpkiss | 16:40 |
| jeblair | mordred: yep. i have a gertty running right now. | 16:40 |
| mordred | jeblair: awesome. so, am I just dumb for thinking that "curl https://review.openstack.org/groups" should do something? | 16:41 |
| jeblair | mordred: you want https://review.openstack.org/groups/ | 16:42 |
| jeblair | mind that trailing / | 16:42 |
| mordred | ah | 16:42 |
| mordred | ok. so that needs auth - but that's fine - I'll just look at gertty to see how you do that | 16:42 |
| derekh | btw, this hasn't merged yet so may be still causing problems https://review.openstack.org/#/c/90757/ | 16:42 |
| fungi | mordred: and you probably need to set yourself up with an http password in your preferences | 16:43 |
| mordred | gotcha | 16:43 |
| bnemec | Any idea why Jenkins doesn't want to recheck https://review.openstack.org/#/c/85224/ ? | 16:43 |
| derekh | fungi: mordred ^ is that what ye are talking about in the get-pip.py url | 16:43 |
| fungi | derekh: oh, that's the other one i thought i saw for that. wonder why it hasn't merged... | 16:43 |
| *** markmcclain has quit IRC | 16:44 | |
| derekh | fungi: beats me | 16:44 |
| fungi | derekh: i think its initial check job got lost in yesterday's restarts, so the approval with no existing verify vote didn't trigger anything | 16:46 |
| fungi | derekh: i just did a recheck no bug on it and it's got jobs runnin gnow | 16:46 |
| fungi | fingers crossed | 16:46 |
| mordred | derekh: yes | 16:46 |
| derekh | fungi: ok, thanks | 16:46 |
| fungi | mordred: your 90573 will be much smaller rebased on that | 16:47 |
| mordred | yup | 16:47 |
| *** dangers_away is now known as dangers | 16:47 | |
| openstackgerrit | Michael Krotscheck proposed a change to openstack-infra/storyboard: Stories can be filtered with LIKE clauses https://review.openstack.org/90478 | 16:47 |
| *** vhoward has left #openstack-infra | 16:48 | |
| *** GheRivero has joined #openstack-infra | 16:48 | |
| *** sweston has joined #openstack-infra | 16:49 | |
| anteaya | jeblair mordred so is gertty somewhere that others can consume yet? | 16:50 |
| anteaya | because I am very curious | 16:51 |
| jeblair | anteaya: i'm trying to figure out why puppet isn't running on the git servers, which is required in order to create the repo for it | 16:51 |
| jeblair | (and any other project) | 16:51 |
| anteaya | jeblair: ah | 16:51 |
| zaro | jeblair, mordred : i don't know if you can remove the monitoring menu. i believe it's configured from the javamelody plugin. | 16:51 |
| *** atiwari has joined #openstack-infra | 16:52 | |
| zaro | jeblair, mordred : i believe there is a plugin that will allow you to customize the top menues. real simple plugin | 16:52 |
| *** weshay has quit IRC | 16:52 | |
| mordred | jeblair: mind if I poke on git03 while you're poking on git01? | 16:53 |
| * mordred doens't want to step on jeblair work | 16:54 | |
| zaro | mordred, jeblair : https://github.com/dluksza/gerrit-top-menu-extension-example | 16:54 |
| jeblair | mordred: go for it | 16:55 |
| *** SumitNaiksatam has quit IRC | 16:55 | |
| *** mrmartin has quit IRC | 16:55 | |
| *** krtaylor has quit IRC | 16:56 | |
| *** julim has joined #openstack-infra | 16:57 | |
| zaro | jeblair, mordred : would it make sense to clean the db for review-dev? I've noticed that it somehow got into a state where the schema on review-dev db is slightly different than what's on review db | 16:57 |
| *** julim has quit IRC | 16:58 | |
| jeblair | mordred: the value passed to git_backend has a newline | 16:58 |
| mordred | putting strip() around it makes it work - so yeah, I agree | 16:58 |
| mordred | now where is the newline coming from? | 16:58 |
| jeblair | mordred: i think because of the "|" thing in yaml | 16:59 |
| *** harlowja_away is now known as harlowja | 16:59 | |
| openstackgerrit | Dan Prince proposed a change to openstack-infra/config: Add yum.openstack.org lightweight Fedora 20 mirror https://review.openstack.org/90875 | 16:59 |
| openstackgerrit | Dan Prince proposed a change to openstack-infra/config: Add yum mirror support for lightweight mirrors https://review.openstack.org/90874 | 16:59 |
| *** etoews has quit IRC | 16:59 | |
| mordred | jeblair: I was justa bout to say the same thing | 16:59 |
| anteaya | zaro: is the data in review-dev db a copy of other data? | 16:59 |
| mordred | jeblair: you wanna take it out? | 16:59 |
| jeblair | mordred: done | 17:00 |
| *** julim has joined #openstack-infra | 17:00 | |
| jeblair | mordred: git01 --noop seems to be looking good | 17:01 |
| mordred | ++ | 17:01 |
| mordred | same on git03 | 17:01 |
| zaro | anteaya: no, data is completely difference. schema is not the same either. we should make the schema the same so it's more like prod gerrit. | 17:01 |
| *** derekh has quit IRC | 17:02 | |
| *** hashar has quit IRC | 17:02 | |
| *** zns has quit IRC | 17:02 | |
| jeblair | i think we just missed the 1700 run; when it finishes, i'll kick run_all again | 17:02 |
| *** amcrn has joined #openstack-infra | 17:02 | |
| mriedem | do the images that are used to run devstack/tempest in community have net.core.somaxconn set to something besides the default 128? | 17:02 |
| anteaya | zaro: yes, since the purpose of review-dev is staging for review, the db schema on review-dev should mirror the db schema on review | 17:03 |
| anteaya | any other opinons? | 17:04 |
| mordred | jeblair: awesome | 17:04 |
| *** andreykurilin_ has joined #openstack-infra | 17:04 | |
| *** flaper87 is now known as flaper87|afk | 17:05 | |
| *** Ryan_Lane has joined #openstack-infra | 17:06 | |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/config: Use storyboard for gertty https://review.openstack.org/91102 | 17:07 |
| openstackgerrit | A change was merged to openstack-infra/config: Replace raw.github.com with raw.githubusercontent.com https://review.openstack.org/90757 | 17:08 |
| jesusaurus | yfried: pong | 17:08 |
| openstackgerrit | Monty Taylor proposed a change to openstack-infra/config: Fix ssh key error and also remove duplicates https://review.openstack.org/91066 | 17:08 |
| mordred | puppet-lint ^^ | 17:08 |
| *** flaper87|afk is now known as flaper87 | 17:08 | |
| *** zehicle_at_dell has joined #openstack-infra | 17:08 | |
| *** SumitNaiksatam has joined #openstack-infra | 17:09 | |
| openstackgerrit | Monty Taylor proposed a change to openstack-infra/config: Remove accumulated python cruft https://review.openstack.org/90573 | 17:10 |
| openstackgerrit | Monty Taylor proposed a change to openstack-infra/config: Fix install_puppet.sh for trusty https://review.openstack.org/90574 | 17:10 |
| mordred | fungi: both of those ^^ are tiny little changes - now rebased on top of the github url content change | 17:11 |
| *** mkoderer has quit IRC | 17:12 | |
| lcostantino | qq, xenserver is still failing with security-groups issues right? or it's sporadic? | 17:12 |
| *** signed8bit has joined #openstack-infra | 17:12 | |
| jeblair | mordred: https://review.openstack.org/#/c/88554/ | 17:14 |
| jeblair | that is really impressive... | 17:14 |
| jeblair | i think gertty has managed to upload a review that says it has inline comments (without any), and a review that somehow was merged with zuul's review | 17:15 |
| tjones | another gerrit oddity today. When i search for status:open message:"refactor - phase 1" i get nothing, but when i search for status:open message:"phase 1" i get a bunch of stuff (including what I am looking for). This search used to give me exactly what i wanted. | 17:15 |
| *** rcarrillocruz has quit IRC | 17:15 | |
| mordred | jeblair: I am very impressed by your technical skills and would like to subscribe to your newsletter | 17:16 |
| anteaya | tjones: does this doc about searching help explain your results? https://review.openstack.org/Documentation/user-search.html | 17:16 |
| *** jpich has quit IRC | 17:16 | |
| *** praneshp has joined #openstack-infra | 17:17 | |
| *** morganfainberg_Z is now known as morganfainberg | 17:18 | |
| anteaya | tjones: looks like this library is used for regex matches: http://www.brics.dk/automaton/ | 17:18 |
| anteaya | I wonder if that lib is the same version as we were using before | 17:18 |
| tjones | anteaya: not really. I would still expect message="refactor - phase 1" to narrow down from message="phase 1" | 17:18 |
| *** jooools has quit IRC | 17:19 | |
| *** sacharya has joined #openstack-infra | 17:19 | |
| mordred | tjones: I would too - but I often expect sanity | 17:19 |
| anteaya | there is something different with how double quotes are used in searches | 17:19 |
| anteaya | what do you get if you try message='refactor - phase 1' | 17:20 |
| openstackgerrit | A change was merged to openstack-infra/config: Use storyboard for gertty https://review.openstack.org/91102 | 17:20 |
| *** chandan_kumar has joined #openstack-infra | 17:20 | |
| tjones | ha just tried that. got an error - status:open message:"refactor - phase 1" | 17:20 |
| tjones | oops | 17:20 |
| tjones | the error was no viable alternative at character ''' | 17:21 |
| anteaya | hmmm | 17:21 |
| *** nati_ueno has joined #openstack-infra | 17:21 | |
| *** rcarrillocruz has joined #openstack-infra | 17:21 | |
| jeblair | infra-manual and gertty projects exist now | 17:21 |
| mordred | jeblair: w00t! | 17:21 |
| * mordred goes to add them to his watch list | 17:21 | |
| *** arnaud has joined #openstack-infra | 17:22 | |
| tjones | i wonder if the commit title is not part of the "commit body" any more | 17:22 |
| jeblair | i might not upload gertty just yet, until i can make sure it's not uploading very crappy data to gerrit | 17:22 |
| * anteaya nods | 17:22 | |
| anteaya | and yay for new repos | 17:22 |
| tjones | odd | 17:22 |
| *** gyee has joined #openstack-infra | 17:23 | |
| anteaya | tjones: can you post the url you are hoping your search locates? | 17:23 |
| * jeblair switches back to the tag problem | 17:23 | |
| mordred | jeblair: uhm | 17:23 |
| mordred | jeblair: I do not see openstack-infra/infra-manual in gerrit | 17:23 |
| *** changbl has joined #openstack-infra | 17:23 | |
| jeblair | oh, they exist on git0* | 17:24 |
| tjones | anteaya: here they are https://review.openstack.org/#/q/status:open+message:%22spawn+refactor+-+phase+1%22,n,z https://review.openstack.org/#/q/status:open+message:%22phase+1%22,n,z | 17:24 |
| openstackgerrit | Marc Abramowitz proposed a change to openstack-infra/jenkins-job-builder: Add more tests https://review.openstack.org/90524 | 17:24 |
| mordred | oh - well, that's good at least | 17:24 |
| mordred | jeblair: you switch back to tag problem - I'll look at the projects on review | 17:24 |
| jeblair | oh no they don't | 17:24 |
| jeblair | puppet on git03.openstack.org failed with return code: 1 | 17:24 |
| jeblair | mordred: ^ | 17:24 |
| mordred | jeblair: how about I look at the whole thing | 17:24 |
| jeblair | mordred: git01 and 02 worked but not 03 | 17:24 |
| *** sabari has quit IRC | 17:25 | |
| jeblair | mordred: ++ lemme know if you need anything, i'll work on tags | 17:25 |
| mordred | jeblair: doh | 17:25 |
| mordred | I had left puppet disabled on git03 - my bad | 17:25 |
| anteaya | tjones: okay I get it you are searching the commit message | 17:25 |
| mordred | good to know the system is actually working as designed! | 17:25 |
| *** zns has joined #openstack-infra | 17:25 | |
| jeblair | heh | 17:25 |
| tjones | anteaya: yes - i am really looking for things with the top line of the commit message (akak title) that match "spawn refactor - phase 1" | 17:26 |
| * anteaya nods | 17:26 | |
| anteaya | catching up to you | 17:26 |
| asselin__ | Hi, I'm trying to do a check bug here for multiple bugs, but it didn't kick off a new check. Is this related to the new gerrit or is my comment not correct? https://review.openstack.org/#/c/90449/ | 17:26 |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/config: Allow infra-core to push tags to sandbox https://review.openstack.org/91108 | 17:27 |
| jeblair | mordred, fungi: ^ whoops | 17:27 |
| mordred | jeblair: +@ - fyi - you no longer need the create permission in refs/tags/* | 17:28 |
| mordred | pushSignedTag by itself is all you need | 17:28 |
| jeblair | mordred: true. i suppose i've also given us perms to create branches. that's probably not a bad thing. | 17:28 |
| mordred | but it also will not break anything, and it's for sandbox | 17:28 |
| *** weshay has joined #openstack-infra | 17:28 | |
| mordred | jeblair: well, if it was refs/* yeah | 17:28 |
| jeblair | oh, i'll fix then | 17:29 |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/config: Allow infra-core to push tags to sandbox https://review.openstack.org/91108 | 17:29 |
| anteaya | tjones: the second search gets you any commit message with the word phase/Phase anywhere in the message | 17:29 |
| anteaya | tjones: it doesn't seem to be doing anything with the 1 | 17:29 |
| tjones | anteaya: ugh | 17:29 |
| tjones | anteaya: so it really does not like multiple words any more i am guessing | 17:30 |
| mordred | wow. that's ... great | 17:32 |
| jeblair | tjones: you might want to ask over in #gerrit | 17:32 |
| *** openstackgerrit has quit IRC | 17:32 | |
| mordred | fungi: can I get some love on https://review.openstack.org/#/c/91066/ ? | 17:33 |
| *** openstackgerrit has joined #openstack-infra | 17:33 | |
| openstackgerrit | Marc Abramowitz proposed a change to openstack-infra/jenkins-job-builder: Add more tests https://review.openstack.org/90524 | 17:34 |
| jeblair | (now i'm trying to figure out why the sandbox-tag job wasn't created) | 17:34 |
| anteaya | tjones: you seem to be correct: https://review.openstack.org/#/q/status:+open+message:%22spawn%22+AND+message:%22refactor%22+AND+message:%22phase%22+AND+message:%221%22,n,z | 17:35 |
| anteaya | tjones: though I have no idea why https://review.openstack.org/#/c/85387/ gets caught in the search results | 17:35 |
| dims | hmm..."recheck migrations" is not re-triggering turbo-hipster | 17:36 |
| mordred | err: Could not retrieve catalog from remote server: Error 400 on SERVER: Invalid parameter soft_write_failure at /opt/config/production/modules/openstack_project/manifests/puppetmaster.pp:108 on node ci-puppetmaster.openstack.org | 17:37 |
| mordred | sigh | 17:37 |
| mordred | I continue to want errors reported to puppetboard | 17:37 |
| mordred | AaronGr, Hunner: ^^ is there any easy/quick fix in our config so that an agent will actually report back if it gets bogus manifests/ | 17:38 |
| tjones | anteaya: i guess i was just lucky before that it worked | 17:39 |
| *** SumitNaiksatam has quit IRC | 17:39 | |
| anteaya | I wouldn't call being able to search for a string lukc | 17:39 |
| anteaya | I would call it common sense | 17:39 |
| fungi | mordred: the "${title}_keys" block in your change to localuser.pp is just transitional, to clkear out the old entries added by the previous iteration? | 17:39 |
| jeblair | mordred: i feel like this report is missing something too: http://puppetdb.openstack.org/report/jenkins01.openstack.org/d20b16cf72665c41d0bf2442bc487cf0931051fb | 17:39 |
| anteaya | the question I have is why do you have to have one word strings for message now? | 17:39 |
| jeblair | mordred: like, i dunno, running jenkins-jobs? | 17:39 |
| *** ativelkov_away is now known as ativelkov | 17:40 | |
| openstackgerrit | Monty Taylor proposed a change to openstack-infra/config: Fix soft_write_failure parameter name. https://review.openstack.org/91109 | 17:40 |
| tjones | yes that is a good question | 17:40 |
| mordred | fungi: yes | 17:41 |
| mordred | jeblair: did it run jenkins jobs and fail? | 17:41 |
| *** SumitNaiksatam has joined #openstack-infra | 17:41 | |
| openstackgerrit | A change was merged to openstack-infra/config: Allow infra-core to push tags to sandbox https://review.openstack.org/91108 | 17:42 |
| fungi | mordred: funny that they refer to the comment on a key as a key id... it's really just a freeform comment area at the end of the line | 17:42 |
| mordred | fungi, jeblair ^^ above change fixes the fact that puppet doesn't run on puppetmaster currently | 17:42 |
| jeblair | mordred: dunno, i'm currently translating PDT to UTC so i can find the right report yaml file on disk | 17:42 |
| mordred | jeblair: lovely | 17:42 |
| jeblair | mordred: i dare you to ask me how i feel about that being the process to find errors. | 17:42 |
| mordred | jeblair: hey man! how do you feel about the need to translate PDT to UTC to that you can find a file on disk to find an error??? **bunny rabbits** | 17:43 |
| anteaya | tjones: one word messages with no quotes nets the same result as one word messages with double quotes: https://review.openstack.org/#/q/status:open+message:spawn+AND+message:refactor++AND+message:phase+AND+message:1,n,z | 17:43 |
| anteaya | tjones: I am suspecting something is going on with the quotes | 17:43 |
| anteaya | but I don't know what yet | 17:43 |
| openstackgerrit | A change was merged to openstack-infra/config: Fix ssh key error and also remove duplicates https://review.openstack.org/91066 | 17:44 |
| jeblair | message: "Failed to call refresh: Command exceeded timeout at /opt/config/production/modules/jenkins/manifests/job_builder.pp:50" | 17:44 |
| tjones | anteaya: that query gives 1 (the 1st one) that does not even have the work spawn in it. something very strange is going on | 17:45 |
| mordred | jeblair: spectacular | 17:45 |
| jeblair | mordred: look, an error! ^ | 17:45 |
| mordred | why would that not be reported to puppetdb? | 17:45 |
| mordred | I mean - it's not puppet failing to compile | 17:45 |
| mordred | (which I disagree with not being reported, but I could at least understand how operationally something might short circuit) | 17:45 |
| jeblair | mordred: i should have known there would be an error because no error was reported to puppetdb and that is how we tell there's an error. | 17:45 |
| mordred | yah | 17:45 |
| jeblair | mordred: istr that we found that some things were not making it to puppetdb, and no one could figure out why. | 17:46 |
| jeblair | mordred: i think this may have been one of them. | 17:46 |
| anteaya | here is the double quotes in a username bug: http://code.google.com/p/gerrit/issues/detail?id=2602 | 17:46 |
| anteaya | tjones: yes, that one patch keeps showing up in every query I try, don't know why | 17:47 |
| *** SumitNaiksatam has quit IRC | 17:48 | |
| *** chandan_kumar has quit IRC | 17:49 | |
| openstackgerrit | Salvatore Orlando proposed a change to openstack-infra/elastic-recheck: Add a fingeprint for bug 1314313 https://review.openstack.org/91112 | 17:50 |
| uvirtbot | Launchpad bug 1314313 in neutron "Firewall fails to become active within 300 seconds" [Critical,New] https://launchpad.net/bugs/1314313 | 17:50 |
| *** mrmartin has joined #openstack-infra | 17:51 | |
| jeblair | oh, the jenkins-jobs runs from earlier are still running | 17:51 |
| jeblair | for about 80 minutes now | 17:51 |
| jeblair | i'd really love to know what they are doing | 17:52 |
| openstackgerrit | A change was merged to openstack-infra/config: Fix soft_write_failure parameter name. https://review.openstack.org/91109 | 17:53 |
| *** ativelkov has left #openstack-infra | 17:53 | |
| anteaya | tjones: not exact but it does give some info on message: http://code.google.com/p/gerrit/issues/detail?id=1441&q=search%20message%3A&colspec=ID%20Type%20Stars%20Milestone%20Status%20Priority%20Owner%20Summary | 17:54 |
| mordred | jeblair: that seems like a long time | 17:54 |
| mordred | jeblair: btw - gertty and infra-manuals are now in gerrit | 17:54 |
| *** sarob has joined #openstack-infra | 17:55 | |
| *** etoews has joined #openstack-infra | 17:56 | |
| tjones | anteaya: i tried topic but that does not work either :-( | 17:56 |
| Hunner | mordred: The agent *should* send a report with the failure messages... is it not? | 17:57 |
| *** markwash has joined #openstack-infra | 17:57 | |
| anteaya | tjones: https://review.openstack.org/#/q/status:open+topic:%255Ebp/vmware-spawn.*,n,z | 17:58 |
| anteaya | apparently strings are very upsetting to search, where as regex gets through: status:open topic:^bp/vmware-spawn.* | 17:59 |
| *** darraghb has quit IRC | 17:59 | |
| anteaya | tjones: is that enough to keep you moving forward? | 17:59 |
| tjones | anteaya: ok i see now. nice. thanks for your help!!! searching strings upsetting…. who knew ;-) | 17:59 |
| anteaya | np | 17:59 |
| mordred | Hunner: yes. the agent sends that report | 18:00 |
| anteaya | yeah, uh, wow is all I can say | 18:00 |
| mordred | it's happily on the puppetmaster in the yaml reports dir | 18:00 |
| mordred | Hunner: but it most certainly does not show up in puppetboard | 18:00 |
| anteaya | tjones: note the . in the .* is part of the regex | 18:00 |
| tjones | anteaya: yeah got it! | 18:00 |
| mordred | Hunner: in a couple of different scenarios we've hit today | 18:00 |
| anteaya | kk | 18:00 |
| *** jp_at_hp has quit IRC | 18:01 | |
| *** etoews has quit IRC | 18:01 | |
| *** _nadya_ has joined #openstack-infra | 18:01 | |
| *** ildikov has joined #openstack-infra | 18:02 | |
| anteaya | there don't seem to be any bug reports about string searching failing with other people's gerrits | 18:02 |
| anteaya | did we have do create any config around searching for our upgrade? | 18:03 |
| mordred | Hunner: one example is: | 18:03 |
| mordred | 17:37:24 mordred | err: Could not retrieve catalog from remote server: Error 400 on SERVER: Invalid parameter soft_write_failure at | 18:03 |
| mordred | | /opt/config/production/modules/openstack_project/manifests/puppetmaster.pp:108 on node ci-puppetmaster.openstack.org | 18:03 |
| *** jamielennox|away is now known as jamielennox | 18:03 | |
| mordred | that manifested as the agent just not reporting at all, which is easy enough to find in puppetboard- "why is that server grey?" - although it does require ability to shell to the server and run commands to figure out why | 18:03 |
| *** mrmartin has quit IRC | 18:04 | |
| *** mengxd has joined #openstack-infra | 18:05 | |
| Hunner | Perhaps puppetboard just doesn't render those reports? If they're submitted to the master, then they should be sent to the puppetdb too... I'll ask | 18:05 |
| jeblair | hrm. jjb gets each job 3 times before posting it | 18:05 |
| mordred | Hunner: there were also 2 other things we ran in to where there was an error reported on teh command line but it didn't show up in puppetboard | 18:06 |
| *** jaypipes has joined #openstack-infra | 18:06 | |
| salv-orlando | there is a rise in neutron unclassified gate failures. A change in a test uncovered a bug, which is now causing failures in the gate: | 18:06 |
| *** jgrimm has quit IRC | 18:06 | |
| mordred | Hunner: and in those cases, puppetboard showed content - it got the report - and even worse, showed that the puppet run was successful | 18:06 |
| salv-orlando | Bug: https://bugs.launchpad.net/neutron/+bug/1314313; e-r query: https://review.openstack.org/#/c/90798/ | 18:06 |
| uvirtbot | Launchpad bug 1314313 in neutron "Firewall fails to become active within 300 seconds" [Critical,New] | 18:06 |
| *** BadCub has quit IRC | 18:06 | |
| *** che-arne has quit IRC | 18:07 | |
| jeblair | jjb appears to be ignoring its cache | 18:07 |
| jeblair | zaro: ^ | 18:07 |
| Hunner | mordred: Do you have an example of that error? | 18:07 |
| Hunner | (the one that did show up and the run was reported successful) | 18:08 |
| mordred | Hunner: yup | 18:08 |
| jeblair | zaro, mgagne: ^ our jjb processes are updating all of the jobs every time they run (even with no changes) | 18:08 |
| mordred | Hunner: look at this: http://puppetdb.openstack.org/report/jenkins01.openstack.org/075bb6a3c6a6593e525a92716e852f62cd27d9c1 | 18:08 |
| *** melwitt has joined #openstack-infra | 18:08 | |
| mordred | Hunner: now, in the report.yaml file, you'll see this: message: "Could not evaluate: Puppet::Util::FileType::FileTypeFlat could not write /home/corvus/.ssh/authorized_keys: Permission denied - /home/corvus/.ssh/authorized_keys" | 18:08 |
| mgagne | jeblair: we cannot be sure often those jobs got updated :D | 18:08 |
| mgagne | jeblair: any idea on the root cause? | 18:08 |
| jeblair | mgagne: not yet | 18:09 |
| anteaya | salv-orlando: thank you for reporting | 18:09 |
| jeblair | (btw, we have 2289 jenkins jobs now) | 18:09 |
| *** mengxd has quit IRC | 18:11 | |
| jeblair | phschwartz: did you have this problem recently? | 18:12 |
| phschwartz | jeblair: Yes I have been having the same issue with the latest jjb | 18:12 |
| jeblair | phschwartz: make any headway? | 18:12 |
| *** etoews has joined #openstack-infra | 18:12 | |
| *** ramashri has joined #openstack-infra | 18:13 | |
| phschwartz | jeblair: I think I haven't narrowed down an actual issue as the source points to a valid location for the cache and it is there. | 18:13 |
| *** BadCub01 has joined #openstack-infra | 18:13 | |
| phschwartz | (or I should say was there after I fix perms on the dir) | 18:14 |
| jeblair | phschwartz: ok, thanks. i'll see what i can come up with | 18:14 |
| phschwartz | I think my issue was something ran as sudo that changed the perms on my cache dir | 18:14 |
| phschwartz | I am doing another run of jjb in a bit so I would have more info | 18:14 |
| *** msabramo has quit IRC | 18:16 | |
| *** etoews has quit IRC | 18:17 | |
| fungi | mordred: hpcloud region a is "west" and region b is "east" right? | 18:17 |
| mordred | fungi: not _quite_ | 18:18 |
| mordred | fungi: or - maybe it is | 18:18 |
| *** etoews has joined #openstack-infra | 18:18 | |
| mordred | yeah - region-a is west - and we don't have it enabled because they told us to stay off it for a while | 18:18 |
| zaro | jeblair: i'll do some testing with jjb locally to see if i can repro. | 18:19 |
| *** talluri has quit IRC | 18:19 | |
| *** _nadya_ has quit IRC | 18:19 | |
| fungi | mordred: well, 1.0 uses the region a endpoint as well | 18:19 |
| fungi | mordred: i guess they share a keystone? | 18:20 |
| openstackgerrit | Salvatore Orlando proposed a change to openstack-infra/elastic-recheck: Add a fingerprint for bug 1313794 https://review.openstack.org/90798 | 18:20 |
| uvirtbot | Launchpad bug 1313794 in neutron "lock_wait timeout caused by plug_vip_port" [High,New] https://launchpad.net/bugs/1313794 | 18:20 |
| mordred | fungi: /me waves hands in the air and then hides | 18:20 |
| fungi | heh | 18:20 |
| *** alexpilotti has joined #openstack-infra | 18:21 | |
| *** jgrimm has joined #openstack-infra | 18:24 | |
| anteaya | just joined the #gerrit channel, channel topic identifies 2.7 | 18:24 |
| zaro | phschwartz, jeblair : this one merged a while ago but might be the cause? https://review.openstack.org/#/c/59515/ | 18:25 |
| phschwartz | zaro: I was just looking at that commit directly to see if it is causing an issue | 18:26 |
| zaro | phschwartz, jeblair : this one as well because it updates how the config file gets loaded. https://review.openstack.org/#/c/52080 | 18:26 |
| mordred | zaro, phschwartz that LOOKS like would wind up with ignore_cache being false in most cases appropriately | 18:27 |
| jeblair | zaro: we don't set that, and it looks like it defaults to false | 18:27 |
| zaro | that one merged just last week. | 18:27 |
| phschwartz | Yeah, after just stepping through a jjb run in pdb, it is set to false and it is trying to load the cache so I am even more stumped. | 18:27 |
| *** hashar has joined #openstack-infra | 18:28 | |
| anteaya | just when through the gerrit irc logs for april, noone has used the word "string" that I could find | 18:28 |
| anteaya | http://echelog.com/logs/browse/gerrit/1396303200 | 18:28 |
| mordred | I could imagine a world in which description mismatches would cause it to think it's changed | 18:29 |
| mordred | gah | 18:29 |
| mordred | I actually re-read that patch and it also looks sane to me | 18:29 |
| *** manishg has joined #openstack-infra | 18:30 | |
| *** zns has quit IRC | 18:31 | |
| zaro | phschwartz: i assume you are using jjb from master? | 18:31 |
| phschwartz | lastest from pip | 18:31 |
| phschwartz | and master | 18:31 |
| *** pdmars has quit IRC | 18:33 | |
| phschwartz | ok, so now I can see after a few tests. My issue was purely due to dir perms where my user couldn't write to the dir. It is working from cache with no issue after the perms fix. | 18:33 |
| mordred | jeblair: ^^ any chance we have a perms issue? | 18:34 |
| yfried | jesusaurus: Hi, I'm shipping files to logstash during debug | 18:35 |
| phschwartz | mordred: my perm issue was the dir I was trying to use as cache was owned by root due to another app that I ran with sudo | 18:35 |
| *** manishg_ has joined #openstack-infra | 18:35 | |
| yfried | jesusaurus: and when I'm trying to reship some files it ignores it, even though I've specificly deleted all sincedb files on the shipper machine | 18:36 |
| *** talluri has joined #openstack-infra | 18:36 | |
| *** msabramo has joined #openstack-infra | 18:36 | |
| *** manishg has quit IRC | 18:37 | |
| *** manishg_ is now known as manishg | 18:37 | |
| jesusaurus | yfried: are you setting start_position? by default it starts at the end so you will need to append lines to the file before anything goes through logstash | 18:38 |
| manishg | I'm getting 503 when accessing review.openstack.org since yesterday (when using firefox) - it started yesterday (was working fine earlier). works fine with safari. | 18:38 |
| yfried | jesusaurus: I am | 18:39 |
| sdague | manishg: did you close your browser? | 18:39 |
| mordred | SergeyLukjanov: you know stackalytics seems to be down? | 18:39 |
| *** melwitt has quit IRC | 18:40 | |
| DinaBelova | mordred, it is ~30 mins | 18:40 |
| manishg | sdague: yeah I closed firefox, I flushed the cache too (yesterday). Let me do it again. | 18:40 |
| *** marcoemorais1 has joined #openstack-infra | 18:40 | |
| yfried | they were shipped ok once. and again after deleting the sincedb files. the third time doesn't work | 18:40 |
| mordred | DinaBelova: ah - ok. cool | 18:40 |
| mordred | just making sure | 18:40 |
| *** melwitt has joined #openstack-infra | 18:41 | |
| manishg | sdague: closed the window but it was still running in background! killed that process and restarted. works fine now. thanks. | 18:41 |
| yfried | jesusaurus: any suggestion? | 18:42 |
| *** AaronGr has quit IRC | 18:42 | |
| *** marcoemorais has quit IRC | 18:42 | |
| sdague | manishg: no prob. now just pay it forward to others asking the question :) | 18:43 |
| manishg | :) sdague: will do! | 18:43 |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/jenkins-job-builder: Treat ignore_cache as a boolean https://review.openstack.org/91124 | 18:45 |
| jeblair | zaro, mgagne, phschwartz, mordred: ^ | 18:45 |
| jesusaurus | yfried: you could add a file output to see if logstash is reading and shipping the events but elasticsearch is failing to index them | 18:45 |
| *** gabriel-bezerra has left #openstack-infra | 18:45 | |
| anteaya | okay we are 15 minutes to -infra meeting, going to try to go for my walk after the meeting then | 18:45 |
| yfried | jesusaurus: it is | 18:45 |
| yfried | jesusaurus: as I said - it worked twice before | 18:46 |
| jesusaurus | and you stopped logstash before deleting the sincedb file? | 18:47 |
| phschwartz | jeblair: Not sure if that is needed, but can't hurt. | 18:47 |
| jesusaurus | yfried: sorry, im not really sure what could be happening | 18:48 |
| yfried | jesusaurus: tnx anyway | 18:48 |
| jesusaurus | yfried: if the logstash process isnt logging any errors and elasticsearch isnt logging any errors, its pretty hard to tell what might be going wrong :/ | 18:48 |
| mgagne | jeblair: would it be a "good idea" to use oslo.config instead? | 18:50 |
| jeblair | phschwartz: it's required -- that's the reason why our runs were not using the cache | 18:50 |
| *** AaronGr has joined #openstack-infra | 18:50 | |
| jeblair | phschwartz: my guess is that if your runs were using the cache, you were probably setting it explicitly on the command line | 18:51 |
| yfried | jesusaurus: it has to do with going over files for the second time | 18:51 |
| jeblair | mgagne: i'm not sure -- i don't really want to add unnecessary dependencies, but if we have mary more command-line-or-config-file options it might be a good idea | 18:55 |
| openstackgerrit | Marc Abramowitz proposed a change to openstack-infra/jenkins-job-builder: Add additional yaml/xml tests, literalinclude them https://review.openstack.org/91131 | 18:57 |
| openstackgerrit | Andreas Jaeger proposed a change to openstack-infra/config: Publish translated localized API documents https://review.openstack.org/91132 | 18:57 |
| openstackgerrit | A change was merged to openstack-infra/jenkins-job-builder: Treat ignore_cache as a boolean https://review.openstack.org/91124 | 18:58 |
| jeblair | meeting time in #openstack-meeting | 18:58 |
| *** hashar is now known as hasharTechTalk | 18:58 | |
| *** pblaho has joined #openstack-infra | 18:59 | |
| jeblair | fungi: around for meeting? | 18:59 |
| fungi | yep | 19:00 |
| *** derekh has joined #openstack-infra | 19:00 | |
| SergeyLukjanov | mordred, yup, looks like stackalytics is down now, maintainers already know about it | 19:00 |
| *** morganfainberg is now known as morganfainberg_Z | 19:01 | |
| *** julim_ has joined #openstack-infra | 19:01 | |
| *** ominakov has joined #openstack-infra | 19:01 | |
| *** ominakov has quit IRC | 19:01 | |
| dansmith | mordred: gerrit (via git-review) is still definitely faster.. dunno if it's gerrit performance or the new box, but... +2 on that from me :) | 19:03 |
| *** signed8bit has quit IRC | 19:03 | |
| dansmith | git-review completes on the same day as I start it now, which is super nice | 19:03 |
| *** julim has quit IRC | 19:04 | |
| openstackgerrit | Michael Krotscheck proposed a change to openstack-infra/storyboard-webclient: Return after successfull auth https://review.openstack.org/90308 | 19:04 |
| *** vhoward has joined #openstack-infra | 19:05 | |
| openstackgerrit | Sergey Lukjanov proposed a change to openstack-infra/config: Add slukjanov to users https://review.openstack.org/91012 | 19:05 |
| openstackgerrit | Marc Abramowitz proposed a change to openstack-infra/jenkins-job-builder: Some tweaks to get closer to Python 3 compat https://review.openstack.org/87810 | 19:07 |
| krotscheck | sdague: When you get a chance, can you look at storyboard.openstack.org, try to edit a task, and see if that typeahead functionality is more like what you were trying to do with the chosen patch? | 19:07 |
| *** vhoward has left #openstack-infra | 19:07 | |
| openstackgerrit | Ivan Melnikov proposed a change to openstack-dev/hacking: Consider sys.real_prefix for _get_import_type https://review.openstack.org/82495 | 19:07 |
| sdague | krotscheck: well, at least in chrome, it's not very clear about what's going in that field | 19:08 |
| krotscheck | sdague: Oh, point. i should add an icon indicator... | 19:08 |
| sdague | yeh, the chosen UI is good in that it shows what you type and the search list | 19:09 |
| sdague | that seems to be jumping to diffent items, but as I don't see my typing, it's pretty blind | 19:09 |
| *** talluri has quit IRC | 19:09 | |
| *** signed8bit has joined #openstack-infra | 19:16 | |
| *** andreykurilin_ has quit IRC | 19:17 | |
| *** SumitNaiksatam has joined #openstack-infra | 19:18 | |
| *** zns has joined #openstack-infra | 19:18 | |
| DinaBelova | mordred - stackalytics is up | 19:18 |
| openstackgerrit | Marc Abramowitz proposed a change to openstack-infra/jenkins-job-builder: Some tweaks to get closer to Python 3 compat https://review.openstack.org/87810 | 19:25 |
| sdague | anteaya: http://lucene.apache.org/core/3_0_3/api/all/org/apache/lucene/analysis/standard/StandardTokenizer.html - some details on what lucene does to text before indexing it | 19:28 |
| *** SumitNaiksatam has quit IRC | 19:28 | |
| sdague | so basically, not being able to find '-' is probably expected | 19:28 |
| anteaya | sdague: thanks | 19:29 |
| *** eharney has joined #openstack-infra | 19:30 | |
| *** mark0n has joined #openstack-infra | 19:31 | |
| *** ociuhandu has quit IRC | 19:32 | |
| notmyname | mordred: what's the status of the m-p merge job? it doesn't look like it ever ran | 19:38 |
| *** ramashri has quit IRC | 19:38 | |
| notmyname | mordred: is there something I can do on my end? | 19:38 |
| *** BadCub01 has quit IRC | 19:38 | |
| zaro | anteaya: i'm not about redoing schema, wanted to see if others wanted to do it first. | 19:39 |
| anteaya | notmyname: when you type m-p I think manage projects since we have been working on that as well | 19:39 |
| anteaya | notmyname: you mean milestone-proposed, yeah? | 19:39 |
| notmyname | yes | 19:39 |
| anteaya | zaro: oh | 19:39 |
| zaro | fungi, mordred, jeblair do you guys think it would be a good idea to sync up review-dev.o.o db schema with one from review.o.o? i think they are out of sync. | 19:40 |
| jeblair | zaro: go for it | 19:40 |
| *** ramashri has joined #openstack-infra | 19:40 | |
| *** nati_ueno has quit IRC | 19:42 | |
| zaro | can someone export the review.o.o schema for me? or should i just let gerrit create fresh one using the gerrit.war? | 19:42 |
| jeblair | zaro: you need to keep the data | 19:43 |
| *** etoews has quit IRC | 19:43 | |
| zaro | jeblair: ohh well, i'm not sure how to sync shema without blowing away data. | 19:43 |
| *** etoews has joined #openstack-infra | 19:44 | |
| zaro | is that even possible? | 19:44 |
| jeblair | zaro: with alter table | 19:45 |
| notmyname | jeblair: do you know anything about the milestone-proposed merge jobs? | 19:45 |
| *** ihrachyshka has quit IRC | 19:45 | |
| jeblair | notmyname: not off hand; is it urgent such that i should try to catch myself up or can it wait for mordred? | 19:46 |
| notmyname | jeblair: it can probably wait until mordred gets back, but pbr versions are not correct until it happens | 19:47 |
| fungi | notmyname: talking about the failure at https://jenkins.openstack.org/job/swift-merge-release-tags/1/console | 19:48 |
| openstackgerrit | Doug Hellmann proposed a change to openstack-infra/config: Import oslo.i18n https://review.openstack.org/91149 | 19:48 |
| zaro | jeblair: ok, whichever way don't i still need an export of the schema from review.o.o ? | 19:50 |
| jeblair | zaro: how do you know it's out of sync? | 19:50 |
| zaro | jeblair: because there's a table on review-dev db that wasn't in the review db. that's why we didn't have you that TRUNCATE workaround in the upgrade db script. | 19:51 |
| zaro | have you/ have to use | 19:52 |
| *** morganfainberg_Z is now known as morganfainberg | 19:52 | |
| jeblair | zaro: i won't be able to get to it for a bit | 19:53 |
| zaro | jeblair: ok. no hurry i guess. whenever. | 19:53 |
| zaro | fungi, jeblair: what needs to happen for review-dev to use the same version of gerrit as review.o.o? i think puppet files have been updated, is it just a restart? | 19:57 |
| *** ociuhandu has joined #openstack-infra | 19:58 | |
| *** krtaylor has joined #openstack-infra | 19:58 | |
| fungi | zaro: probably...? | 19:59 |
| *** hasharTechTalk is now known as hashar | 19:59 | |
| *** ramashri has quit IRC | 20:00 | |
| openstackgerrit | Doug Hellmann proposed a change to openstack-infra/config: Import oslo.i18n https://review.openstack.org/91149 | 20:01 |
| zaro | fungi: how can i check whether puppet has (or has not) updated review-dev? | 20:01 |
| *** pblaho has quit IRC | 20:02 | |
| notmyname | fungi: yes, that one | 20:02 |
| *** derekh has quit IRC | 20:05 | |
| anteaya | tjones: https://review.openstack.org/#/q/message:a_test_title_to.see.what.I.can,n,z | 20:08 |
| *** fifieldt has quit IRC | 20:08 | |
| *** fifieldt has joined #openstack-infra | 20:09 | |
| tjones | anteaya: nice! | 20:09 |
| anteaya | tjones: so it seems to consider words joined by underscores or . as one token | 20:09 |
| *** sarob has quit IRC | 20:09 | |
| anteaya | it really hates the ! even in regexs | 20:09 |
| *** nati_ueno has joined #openstack-infra | 20:09 | |
| anteaya | and it treats - like whitespace | 20:10 |
| *** etoews_ has joined #openstack-infra | 20:10 | |
| anteaya | in terms of splitting on it | 20:11 |
| JayF | Hmm. Ever since the gerrit upgrade, all emails from gerrit are going to my spam folder | 20:11 |
| *** sdake_ has joined #openstack-infra | 20:11 | |
| openstackgerrit | A change was merged to openstack-dev/hacking: Consider sys.real_prefix for _get_import_type https://review.openstack.org/82495 | 20:12 |
| *** Ajaeger has quit IRC | 20:13 | |
| anteaya | ArxCruz krtaylor do either of you know who IBM powerVC is? They need to comment once per patchset, right now they are commenting twice: https://review.openstack.org/#/c/91154/ | 20:13 |
| *** etoews has quit IRC | 20:13 | |
| *** nati_uen_ has joined #openstack-infra | 20:13 | |
| tjones | anteaya: thanks for looking into that | 20:13 |
| anteaya | tjones: np, so the tl;dr is that searching for strings with multiple words won't work anymore | 20:13 |
| ArxCruz | anteaya: I know, but they will be on only in a few hours | 20:14 |
| anteaya | so creative use of specialized tokens, regexs and topic look like it might help you | 20:14 |
| anteaya | ArxCruz: k, I'll leave it with you, thanks | 20:14 |
| *** mattymo|home has quit IRC | 20:16 | |
| *** nati_ueno has quit IRC | 20:17 | |
| ArxCruz | anteaya: already sent an email to the responsible, but they will be available only in 6-7 hours | 20:17 |
| anteaya | ArxCruz: not a big deal, they are not voting on anything, they are just on the sandbox repo right now that I can see | 20:18 |
| anteaya | so it is good they are staying there for now | 20:18 |
| anteaya | ArxCruz: https://review.openstack.org/#/q/owner:pwrdrvci%2540cn.ibm.com+status:open,n,z | 20:19 |
| asselin__ | anyone run into this error when creating a new image? INFO:nodepool.image.build.local.d-p-c:Parameter key failed on Ssh_authorized_key[jenkins-master-2014-04-24]: Key must not contain whitespace: | 20:19 |
| ArxCruz | anteaya: by the way, I'm noticing my user are not posting back the results | 20:19 |
| ArxCruz | anteaya: in my zuul log I see this: | 20:19 |
| asselin__ | (this is my own local setup) | 20:19 |
| ArxCruz | 2014-04-29 20:01:17,503 DEBUG zuul.reporter.gerrit.Reporter: Report change <Change 0x7f7d10ac2310 91146,1>, params {'force-message': True}, message: Build failed. For rechecking only on the IBM PowerKVM CI, add a review comment with recheck-pkvm | 20:20 |
| ArxCruz | but I'm not seeing in review.openstack.org | 20:20 |
| anteaya | ArxCruz: hmmm, you updated host keys | 20:20 |
| ArxCruz | yes... | 20:20 |
| jeblair | ArxCruz: I7836860f9e62bcb169bc13c5e3b5b5ecf6d325a1 | 20:20 |
| jeblair | ArxCruz: sorry, rather this one: I7d1abe590dc79edde4d3ccf998ede59fccc4844a | 20:21 |
| jeblair | ArxCruz: you'll need zuul master as well | 20:21 |
| ArxCruz | ohhh | 20:21 |
| ArxCruz | need to update zuul? | 20:21 |
| ArxCruz | last time I did was so much pain... | 20:21 |
| jeblair | ArxCruz: yes, tip of zuul master plus that change to your layout | 20:21 |
| *** jlk has quit IRC | 20:23 | |
| *** wendar has quit IRC | 20:24 | |
| *** wendar has joined #openstack-infra | 20:24 | |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/config: Drop the ref from the tag fetch in GGP https://review.openstack.org/91164 | 20:25 |
| ArxCruz | jeblair: okay, thanks | 20:25 |
| jeblair | mordred, fungi: ^ that should take care of the tag problem | 20:25 |
| anteaya | asselin__: so in the backscroll jeblair encountered an issue with a newline in a key, he wrapped the value in strip() to remove it, I don't think there was a patch | 20:25 |
| anteaya | asselin__: but I could be wrong | 20:25 |
| fungi | okay, well i seem to be able to rebuild bare-centos6 images successfully now, and the count seems to be better distributed across providers again | 20:25 |
| jeblair | mordred, fungi: the problem likely occurred only while we were having replication issues. the fix above will make it so that it should work regardless of replication status | 20:26 |
| jeblair | mordred, fungi: moreover, i think very rarely we have seen that failure in normal operation -- those cases were likely replication delays which this should fix | 20:26 |
| jeblair | mordred, fungi: (i tested by cloning a copy of sandbox from the old gerrit server; that was helpful) | 20:27 |
| anteaya | asselin__: http://eavesdrop.openstack.org/irclogs/%23openstack-infra/%23openstack-infra.2014-04-29.log timestamp 2014-04-29T16:58:26 | 20:27 |
| jeblair | fungi: what was the contos problem? | 20:27 |
| fungi | jeblair: oh, that's a tricky one. so if you fetch a specific tag it needs to be one your remote update found, but if you fetch all tags it will get them even when remote update failed? | 20:27 |
| jeblair | fungi: (btw, i think you and i are the only cores around if you want to go aheand and aprv https://review.openstack.org/91164) | 20:28 |
| jeblair | fungi: even better -- it only fails the first time. a subsequent fetch will work | 20:28 |
| ArxCruz | jeblair: so gerrit: { } means post a message failing or not, but don't vote ? | 20:28 |
| jeblair | fungi: (because the first fetch got the info needed for the second fetch to work) | 20:28 |
| *** jooools has joined #openstack-infra | 20:28 | |
| jeblair | ArxCruz: yeah, it's basically "use this reporter with no arguments" | 20:28 |
| fungi | jeblair: the centos6 image build issue was curl not following the github redirect to the new url, so that outstanding patch definitely solved it | 20:28 |
| asselin__ | anteaya, thanks...not sure if it's the same...but I will try it | 20:29 |
| jeblair | fungi: what caused the 70 building/deleting nodes? | 20:29 |
| anteaya | asselin__: that is all I have on that one | 20:29 |
| fungi | jeblair: that i'm less sure about. from the logs it looked like we were timing out ssh access to them in hpclolud-region-b and it was going into a tight rebuild loop | 20:30 |
| *** sabari has joined #openstack-infra | 20:30 | |
| openstackgerrit | A change was merged to openstack-infra/config: Drop the ref from the tag fetch in GGP https://review.openstack.org/91164 | 20:30 |
| jeblair | ah | 20:31 |
| fungi | jeblair: it subsided on its own before i could pin it down, so not really sure | 20:32 |
| fungi | asselin__: https://review.openstack.org/91066 | 20:32 |
| asselin__ | fungi, I saw that and tried it but it didn't solve the issue for me...I noticed in another commit the ssh keys had their whitespace and comment section removed. | 20:33 |
| SlickNik | fungi / anteaya: Fixed the issue with the gerrit-trigger plugin. Turns out that I had already updated the host key fingerprint and that wasn't the issue. | 20:34 |
| *** pdmars has joined #openstack-infra | 20:34 | |
| anteaya | SlickNik: what was the issue? | 20:34 |
| SlickNik | fungi / anteaya: The deprecated 'approve' alias for the 'review' SSH command has been removed as of gerrit 2.8. | 20:34 |
| SlickNik | And gerrit trigger plugin was using this alias. | 20:35 |
| anteaya | how did you fix it? | 20:35 |
| asselin__ | fungi, mordred do you know the reason why that change is needed? i.e. a change in puppet? | 20:35 |
| SlickNik | (by default) | 20:35 |
| anteaya | SlickNik: just remove the alias? | 20:35 |
| SlickNik | THey let you override the commands in the global configuration for the plugin though. | 20:35 |
| anteaya | SlickNik: oh you changed it there, cool | 20:36 |
| SlickNik | So I just changed the commands to use 'review' instead of 'approve' | 20:36 |
| anteaya | cool | 20:36 |
| SlickNik | Yup, that seems to be working well. | 20:36 |
| anteaya | thanks for letting us know | 20:36 |
| anteaya | your's won't be the last system tripping on this | 20:36 |
| fungi | asselin__: ahh, yep, https://review.openstack.org/87340 | 20:36 |
| SlickNik | Not a problem. Now you know. :) | 20:37 |
| SlickNik | FYI: The release notes that talk about the removal of the deprecated 'approve' alias - https://gerrit-documentation.storage.googleapis.com/ReleaseNotes/ReleaseNotes-2.8.html | 20:37 |
| fungi | asselin__: to support being able to run puppet agent on demand via ssh from the master we implemented the ssh_authorized_key type, so that change started using it everywhere for consistency | 20:38 |
| *** jlk has joined #openstack-infra | 20:38 | |
| SlickNik | The helpful part is under the 'Schema Change' section | 20:38 |
| fungi | SlickNik: oh, yep, someone else posted a similar note to either the dev or infra ml, i forget which | 20:38 |
| *** yidclare has quit IRC | 20:39 | |
| SlickNik | fungi: Cool, :) Just wanted to give you guys a heads up in case you get hit with a similar question from other folks. | 20:40 |
| asselin__ | fungi, ok so perhaps that broke the instructions in nodepool readme: export NODEPOOL_SSH_KEY=`cat ~/.ssh/id_rsa.pub` | 20:40 |
| anteaya | SlickNik: https://wiki.openstack.org/wiki/GerritUpgrade#Gerrit_Trigger_Plugin | 20:40 |
| asselin__ | fungi, since the public key can have spaces...I'll take a look and see if that is it | 20:40 |
| anteaya | SlickNik: do add anything you think is missing | 20:40 |
| mordred | asselin__: aha! | 20:40 |
| mordred | asselin__: let me go look ... it's possible we broke that | 20:41 |
| asselin__ | mordred, thanks! | 20:41 |
| SlickNik | anteaya: Thanks! That looks good. | 20:42 |
| anteaya | ArxCruz had posted to the infra ml about changing the host key: http://lists.openstack.org/pipermail/openstack-infra/2014-April/001201.html | 20:42 |
| anteaya | SlickNik: thanks | 20:42 |
| fungi | asselin__: as a workaround, you might try switching to NODEPOOL_SSH_KEY=`cut -d' ' -f-2 ~/.ssh/id_rsa.pub` | 20:42 |
| *** julim_ has quit IRC | 20:42 | |
| asselin__ | fungi, thanks will try now | 20:42 |
| mordred | fungi: I think we might want to do that in prepare_node.sh | 20:42 |
| fungi | mordred: we probably need to split it there and pass in the separate vars, agreed | 20:43 |
| mordred | fungi: we could test to see if the ssh key string has ssh-rsa in it, and if so , split it | 20:45 |
| mordred | jeblair: so - apparently we've grown instances of gliffy, the code review cuckoo? | 20:46 |
| jeblair | mordred: 'diffy' | 20:46 |
| *** msabramo has quit IRC | 20:47 | |
| mordred | oh. right. sorry | 20:47 |
| jeblair | mordred: and it appears to be hardcoded in the gwt side of things; completely ignoring the server side avatar configuration | 20:47 |
| jeblair | top notch this thing. | 20:47 |
| anteaya | ha ha ha | 20:47 |
| anteaya | the bird is mandatory | 20:47 |
| jeblair | anteaya: indeed, i feel like the gerrit devs are insisting on giving us the bird. | 20:48 |
| anteaya | ha ha ha | 20:48 |
| mordred | jeblair: well - I betcha we could javascript that bird out of existence | 20:48 |
| anteaya | they can't get the ux right | 20:49 |
| anteaya | but they hard code a troll | 20:49 |
| mordred | which I'm kind of tempted to waste mental energy on just out of general principle | 20:49 |
| jeblair | mordred: yeah; it's not obvious to me at the moment how to do that | 20:49 |
| sdague | jeblair: https://etherpad.openstack.org/p/gerrit-dashboards what do you think of this for a dashboard set | 20:49 |
| fungi | we should totally replace diffy with gliffy the code review troll | 20:50 |
| mordred | fungi: or a picture of zuul | 20:50 |
| mordred | there is no diffy, only zuul | 20:50 |
| anteaya | the dragon in the tutu? | 20:50 |
| fungi | touché | 20:51 |
| jeblair | while i'm ranting -- i checked in on project renames: https://gerrit-review.googlesource.com/#/c/42247/ | 20:51 |
| jeblair | "Updated 8 months ago" | 20:51 |
| mordred | sdague: what does foreach do? | 20:51 |
| sdague | means it can apply to projects as a token | 20:51 |
| sdague | mordred: https://review.openstack.org/Documentation/user-dashboards.html#_token_code_project_code | 20:52 |
| openstackgerrit | Davanum Srinivas (dims) proposed a change to openstack-infra/config: Gerrit - increase the width of the comments https://review.openstack.org/91170 | 20:52 |
| anteaya | sdague: Lost changes makes me thing of patches that have logs that are lost, due to a random zuul error | 20:52 |
| mordred | sdague: those look like good things | 20:52 |
| sdague | so then if you applied it to nova, you'd only get nova code, for instance | 20:52 |
| anteaya | sdague: do you care that I think that? | 20:52 |
| jeblair | sdague: i don't read it as being required in order to use the project token | 20:53 |
| *** adalbas has quit IRC | 20:53 | |
| jeblair | sdague: i read it as shorthand for "append to all queries" | 20:53 |
| sdague | jeblair: ok, yeh, I'm not sure | 20:53 |
| jeblair | (which i worry would make it harder to write non project-specific queries) | 20:53 |
| sdague | anteaya: if you have a better name suggestion, I'm all for it | 20:53 |
| sdague | jeblair: it would be good to see it in operation to understand it's limitations | 20:54 |
| *** _nadya_ has joined #openstack-infra | 20:54 | |
| mordred | so - right now, important changes is a file called "important-changes-dashboard" in the ref refs/dashboards/important-changes on All-Projects | 20:54 |
| anteaya | adrift, lacking, astray, wayward | 20:54 |
| anteaya | I love a trip to the thesaurus | 20:55 |
| anteaya | like any of them? | 20:55 |
| mordred | which is to say - any dashboard needs a category (refs/dashboards/important-changes) and a name "important-changes-dashboard" | 20:55 |
| mordred | I'm not 100% sure how to delete a dashboard ref, btw | 20:55 |
| mordred | but inside of that t's just files | 20:55 |
| openstackgerrit | A change was merged to openstack-infra/elastic-recheck: Add a fingerprint for bug 1313851 https://review.openstack.org/90794 | 20:56 |
| mordred | for testing, since deleting a file is easy - I could push a new additional file to the important-changes ref ? | 20:56 |
| uvirtbot | Launchpad bug 1313851 in neutron "lock wait timeout in create_dhcp_port" [Medium,New] https://launchpad.net/bugs/1313851 | 20:56 |
| sdague | mordred: that seems like an undoable thing to do | 20:57 |
| fungi | mordred: might be able to push a null ref to it. doesn't look like the ui treats those as actual branches though | 20:57 |
| sdague | so if we do it badly it can be wiped out | 20:57 |
| *** e0ne has joined #openstack-infra | 20:57 | |
| mordred | that's my main concern right now | 20:57 |
| sdague | also, who has permissions to do refs/ manip? is that just the manage-projects script? | 20:58 |
| jeblair | sdague, mordred: i think as long as sdague's queries are going in a different file, the foreach should be fine -- if they get combined with the existing important changes dashboard file, i think the project token needs to distribute down to those 3 queries and the foreach should be removed | 20:58 |
| fungi | mordred: since it looks like a branchish ref in git, and it's possible to push :remotename to delete branches, i'd say it's worth trying on review-dev | 20:58 |
| mordred | and project bootstrappers | 20:58 |
| sdague | jeblair: yeh, I think this should be a different file | 20:58 |
| mordred | jeblair: I agree | 20:58 |
| asselin__ | BTW, I have a change that's still needs review. What's the best way to get it reviewed? I'd like some initial feedback before adjusting the merge conflicts. https://review.openstack.org/#/c/87384/ | 20:58 |
| mordred | jeblair, fungi: so - you guys fine with me pushing that dashboard to test it out? | 20:59 |
| jeblair | mordred: yes | 20:59 |
| mordred | done | 21:00 |
| jeblair | mordred: i think you should undo it; i get a 500 | 21:00 |
| anteaya | scant, wanting, missing, scarce | 21:00 |
| mordred | yup | 21:00 |
| *** yidclare has joined #openstack-infra | 21:00 | |
| *** SumitNaiksatam has joined #openstack-infra | 21:00 | |
| mordred | undone | 21:00 |
| jeblair | better | 21:00 |
| mordred | wow. that was exciting | 21:01 |
| sdague | any indication what went wrong? | 21:01 |
| fungi | wow, that's... fun | 21:01 |
| *** melwitt has quit IRC | 21:01 | |
| mordred | nope. I got no errors on push | 21:01 |
| jeblair | i'm thinking review-dev might be a good place to push that next. | 21:01 |
| mordred | jeblair: yup | 21:01 |
| mordred | jeblair: might be worth giving sdague project bootstrappers there so he can just push to his heart's content for a bit? | 21:01 |
| jeblair | mordred: wfm | 21:01 |
| anteaya | sdague: so far I like wayward the best | 21:01 |
| sdague | anteaya: ++ wayward sounds good | 21:02 |
| mordred | sdague: I'm going to put you in the project bootstrappers group on review-dev | 21:02 |
| sdague | mordred: sounds good | 21:02 |
| jeblair | maybe section names | 21:02 |
| *** _nadya_ has quit IRC | 21:02 | |
| sdague | does review-dev have enough data to make this useful? | 21:02 |
| mordred | sdague: https://etherpad.openstack.org/p/gerrit_upgrade_script has a section on adding a new dashboard | 21:02 |
| *** melwitt has joined #openstack-infra | 21:03 | |
| jeblair | sdague: yeah, and you can always add more data | 21:03 |
| sdague | ok | 21:03 |
| mordred | sdague: I think we're going for "functions at all without breaking things" before we try pushing to review again | 21:03 |
| jeblair | mordred: ++ | 21:03 |
| zaro | jeblair: there's an upcoming change to allow hiding monitoring menu: https://gerrit-review.googlesource.com/56598 | 21:03 |
| anteaya | mordred: can I be added to project bootstrappers on review-dev as well? I would like to play with dashboards too | 21:03 |
| *** marcoemorais1 has quit IRC | 21:03 | |
| *** msabramo has joined #openstack-infra | 21:03 | |
| sdague | mordred: I'm good with that | 21:03 |
| mordred | sdague: done | 21:03 |
| *** aysyd has quit IRC | 21:03 | |
| sdague | this sounds like a reasonable plan, I'm happy to be a part of it :) | 21:03 |
| mordred | woot! | 21:04 |
| openstackgerrit | Ben Nemec proposed a change to openstack-infra/reviewstats: Update for Gerrit 2.8 https://review.openstack.org/91173 | 21:04 |
| jeblair | zaro: cool. i'm quite okay with the default of falso, btw. | 21:04 |
| openstackgerrit | A change was merged to openstack-infra/config: update jenkins-dev.o.o with same plugins as jenkins.o.o https://review.openstack.org/77040 | 21:04 |
| *** marcoemorais has joined #openstack-infra | 21:04 | |
| mordred | jeblair: can more things default to falso | 21:05 |
| mordred | ? | 21:05 |
| fungi | mordred: not sure if you saw, but notmyname was asking about https://jenkins.openstack.org/job/swift-merge-release-tags/1/console (looks like maybe git-review wasn't expecting to find another outstanding commit besides the merge commit?) | 21:05 |
| notmyname | mordred: fungi: just wondering about the current status. versions on master are wrong until m-p gets merged. | 21:06 |
| *** marcoemorais has quit IRC | 21:06 | |
| *** andreaf has quit IRC | 21:06 | |
| sdague | mordred: is there a way to actually check out the existing refs/meta/... ? | 21:07 |
| *** marcoemorais has joined #openstack-infra | 21:08 | |
| zaro | sdague: git fetch origin refs/meta/config;git checkout FETCH_HEAD | 21:10 |
| openstackgerrit | A change was merged to openstack-infra/elastic-recheck: Add a fingerprint for bug 1313800 https://review.openstack.org/90797 | 21:10 |
| uvirtbot | Launchpad bug 1313800 in neutron "lock wait timeout while updating LB pool" [Medium,New] https://launchpad.net/bugs/1313800 | 21:10 |
| *** dangers is now known as dangers_away | 21:10 | |
| bknudson | it's not going to break anything if we use the "New" view in gerrit? | 21:11 |
| mordred | bknudson: your sanity | 21:12 |
| bknudson | might be too late for that already | 21:12 |
| sdague | zaro: thanks | 21:13 |
| *** SumitNaiksatam has quit IRC | 21:14 | |
| *** dizquierdo has joined #openstack-infra | 21:15 | |
| *** SumitNaiksatam has joined #openstack-infra | 21:15 | |
| openstackgerrit | A change was merged to openstack-infra/elastic-recheck: Add a fingerprint for bug 1313794 https://review.openstack.org/90798 | 21:17 |
| uvirtbot | Launchpad bug 1313794 in neutron "lock_wait timeout caused by plug_vip_port" [High,New] https://launchpad.net/bugs/1313794 | 21:17 |
| *** dkliban is now known as dkliban_afk | 21:18 | |
| *** krtaylor has quit IRC | 21:18 | |
| *** eharney has quit IRC | 21:19 | |
| openstackgerrit | David Ostrovsky proposed a change to openstack-infra/config: Add configuration option to disable top level menu for javamelody plugin https://review.openstack.org/91176 | 21:20 |
| *** dprince has quit IRC | 21:20 | |
| *** eharney has joined #openstack-infra | 21:21 | |
| *** morganfainberg is now known as morganfainberg_Z | 21:22 | |
| *** hashar has quit IRC | 21:23 | |
| *** morganfainberg_Z is now known as morganfainberg | 21:25 | |
| *** matty_dubs is now known as matty_dubs|gone | 21:26 | |
| sdague | what was the magic exploding url? | 21:28 |
| mordred | sdague: https://review.openstack.org/#/admin/projects/All-Projects,dashboards | 21:28 |
| *** rfolco has quit IRC | 21:29 | |
| sdague | yep, still 500 | 21:29 |
| sdague | can someone see if there is anything useful on the server? | 21:29 |
| sdague | https://review-dev.openstack.org/#/admin/projects/All-Projects,dashboards I mean is 500 | 21:30 |
| openstackgerrit | A change was merged to openstack-infra/config: Add fedora devstack instances to nodepool https://review.openstack.org/86842 | 21:30 |
| *** BadCub has joined #openstack-infra | 21:30 | |
| jeblair | sdague: ack | 21:30 |
| jeblair | sdague: not really: http://paste.openstack.org/show/77809/ | 21:32 |
| *** sdake_ has quit IRC | 21:32 | |
| sdague | hmmm | 21:32 |
| *** morganfainberg is now known as morganfainberg_Z | 21:33 | |
| sdague | oh, description is apparently mandatory | 21:34 |
| *** thomasem has quit IRC | 21:34 | |
| sdague | good to know | 21:34 |
| *** morganfainberg_Z is now known as morganfainberg | 21:34 | |
| openstackgerrit | A change was merged to openstack-infra/config: Add check-rally-dsvm-glance job https://review.openstack.org/89801 | 21:36 |
| jeblair | i'm going to restart gerrit to pick up those changes | 21:36 |
| *** mbacchi has quit IRC | 21:37 | |
| jeblair | dims: ping | 21:38 |
| *** mwagner_lap has quit IRC | 21:39 | |
| sdague | https://review-dev.openstack.org/#/projects/gtest-org/gtest,dashboards/important-changes:main | 21:40 |
| sdague | it's a thing | 21:40 |
| sdague | I had to adjust it because jenkins isn't a user there | 21:41 |
| sdague | but it seems to do the thing we want | 21:41 |
| sdague | jeblair: https://etherpad.openstack.org/p/gerrit-dashboards (the part under Version 2 is the working one) | 21:42 |
| sdague | feel free to change title or description, not wedded to either | 21:43 |
| *** dizquierdo has quit IRC | 21:43 | |
| *** sdake_ has joined #openstack-infra | 21:48 | |
| *** sdake_ has quit IRC | 21:48 | |
| *** sdake_ has joined #openstack-infra | 21:48 | |
| *** lcostantino has quit IRC | 21:48 | |
| jeblair | restarting gerrit now | 21:49 |
| *** praneshp has quit IRC | 21:50 | |
| fungi | k | 21:50 |
| jeblair | done, and the zuul commentlinks look correct | 21:50 |
| *** jeblair changes topic to "Gerrit upgraded to 2.8. See: https://wiki.openstack.org/wiki/GerritUpgrade" | 21:50 | |
| fungi | why yes... yes they do! | 21:50 |
| sdague | jeblair: nice, the css seems to have taken correctly (I can disable stylish on it now) | 21:51 |
| *** smarcet has quit IRC | 21:51 | |
| fungi | i still think the current gerrit design is waaaay to liberal with whitespace (particularly line spacing), but it's usable | 21:51 |
| *** praneshp has joined #openstack-infra | 21:51 | |
| fungi | s/to/too/ | 21:51 |
| jeblair | fungi: agreed | 21:51 |
| fungi | they've injected tons of extra padding for reasons i can't fathom | 21:52 |
| jeblair | fungi: ipad? | 21:52 |
| fungi | maybe they thought the interface was too dense | 21:52 |
| openstackgerrit | Boris Pavlovic proposed a change to openstack-infra/config: Bump check-rally-dsvm-{name} job timeout https://review.openstack.org/90630 | 21:52 |
| fungi | jeblair: heh, so they can finger-paint their links more easily. gotta be it! ;) | 21:53 |
| *** weshay has quit IRC | 21:56 | |
| notmyname | ah. thank you for making the jenkins comments in gerrit readable again | 21:57 |
| *** dims has quit IRC | 21:57 | |
| jeblair | notmyname: ++ | 21:58 |
| jeblair | sdague: "Error in operator label:Code-Review>=0,self" if you aren't logged in | 21:58 |
| jeblair | sdague: that makes sense of course | 21:58 |
| *** mfer has quit IRC | 21:59 | |
| sdague | oh, interesting | 21:59 |
| sdague | I wonder if there is any mitigation for that | 21:59 |
| notmyname | jeblair: I was literally looking through the code to see how to change it. didn't find anything (or actually I think I pulled in the patch before you restarted gerrit and therefore didn't see anything that made sense). then I glanced in here and saw y'all had just fixed it | 21:59 |
| sdague | got to hit dinner, but will be back on after | 22:00 |
| notmyname | sdague: so i take it you are creating generic dashboards for everyone? | 22:00 |
| jeblair | notmyname: yeah, we merged the fix this morning but needed a restart to fix it up | 22:00 |
| openstackgerrit | A change was merged to openstack-infra/elastic-recheck: Add a fingeprint for bug 1314313 https://review.openstack.org/91112 | 22:00 |
| uvirtbot | Launchpad bug 1314313 in neutron "Firewall fails to become active within 300 seconds" [Critical,New] https://launchpad.net/bugs/1314313 | 22:00 |
| fungi | notmyname: the fix was merged ~13 hours ago in https://review.openstack.org/90872 | 22:01 |
| jeblair | sdague: maybe s/main/review-inbox/ ? | 22:01 |
| notmyname | fungi: well that explains why what I found didn't match the DOM I was seeing :-) | 22:01 |
| fungi | notmyname: exactly ;) | 22:01 |
| jeblair | notmyname: yeah, sdague is writing some "zero inbox" style dashboards | 22:02 |
| notmyname | jeblair: nice. | 22:02 |
| jeblair | "Needs final +2", "Passed Jenkins, No Negative Feedback", "Wayward Changes" (basically, changes older than 2d no one has reviewed) | 22:02 |
| jeblair | notmyname: ^ | 22:02 |
| notmyname | cool | 22:03 |
| fungi | ianw: devstack-f20 nodes are building in nodepool currently. i'll let you know if they succeed or need additional troubleshooting | 22:03 |
| notmyname | if my vote counts at all, my (now broken) bookmarks are: "all swift reviews", "patches waiting to merge", "patches with one core reviewer", "open patches I have reviewed", "stuff I haven't reviewed", "patches with no core reviews" | 22:03 |
| *** dosaboy has quit IRC | 22:03 | |
| fungi | notmyname: yeah i too have a query i use for "has at least one +2 and isn't approved yet" | 22:04 |
| *** dosaboy has joined #openstack-infra | 22:04 | |
| notmyname | looks like some of those are being handled already. the "waiting to merge" is pretty usefil, I think | 22:04 |
| jeblair | notmyname: maybe add some notes to: https://etherpad.openstack.org/p/gerrit-dashboards ? | 22:05 |
| notmyname | also, the wayward changes is good, but 2 days is pretty short. | 22:05 |
| jeblair | notmyname: (especially if you can paste in your old queries, and double especially if you can fix them :) | 22:05 |
| notmyname | :-) | 22:05 |
| mattoliverau | Morning everyone | 22:06 |
| jeblair | mattoliverau: good morning | 22:07 |
| *** gondoi is now known as zz_gondoi | 22:08 | |
| fungi | jhesketh_: how has turbo-hipster been holding up since yesterday's gerrit upgrade? i think dims had a patch he added a recheck migrations comment on which t-h never seemed to pick up (though now it has a new patchset, so the -1 is gone anyway) | 22:09 |
| *** dkranz has quit IRC | 22:09 | |
| *** rlandy has quit IRC | 22:09 | |
| *** msabramo has left #openstack-infra | 22:09 | |
| *** dims has joined #openstack-infra | 22:10 | |
| dims | jeblair, pong | 22:11 |
| dims | jeblair, i was staring at http://i.imgur.com/kBCaZhy.png?1 and filed that review. did not realize it was broken urls :) we should be good | 22:11 |
| *** nati_uen_ has quit IRC | 22:12 | |
| dims | jeblair, not sure if this other one is a known problem - comments leak to the right - http://i.imgur.com/XoY1tse.png | 22:12 |
| *** sacharya has quit IRC | 22:17 | |
| jeblair | dims: cool, i was hoping that might be it... as for the other one, i think the old version of gerrit did that too -- i think it may have decided to go into "<pre>", but for that comment, i don't know why :/ | 22:18 |
| *** mfer has joined #openstack-infra | 22:18 | |
| *** afazekas has quit IRC | 22:18 | |
| dims | jeblair, thanks | 22:18 |
| zaro | arrg! gerrit line numbers on right margin is only on old screen, new screen has it on left margin. ohh well. | 22:19 |
| *** mfer has quit IRC | 22:20 | |
| fungi | zaro: maybe they misunderstood the definition of the word "old" | 22:23 |
| zaro | jeblair: ok, new gerrit javamelody plugin build will allow you to hide it's menu. should be available tommorow on tarballs.o.o. I believe you will need to manually install it. | 22:23 |
| *** saschpe has quit IRC | 22:23 | |
| jeblair | zaro: plugin installation isn't under puppet? | 22:24 |
| zaro | jeblair: no, it's not. only the installation of gerrit core + core plugins is in puppet | 22:25 |
| fungi | ianw: image build is failing. full log is at http://nodepool.openstack.org/image.log but i'll see if i can get you some specifics | 22:25 |
| mgagne | Can I say that I feel lost with the "new" screen? :O | 22:25 |
| jeblair | zaro: can you work on fixing that? i thought your puppet changes for plugins supported that. | 22:26 |
| *** saschpe has joined #openstack-infra | 22:26 | |
| jeblair | mgagne: you are in good company here. | 22:27 |
| * mgagne feels better | 22:27 | |
| *** alexpilotti has quit IRC | 22:27 | |
| zaro | jeblair: yeah, i can take a look but i believe gerrit will require a manual restart at least. | 22:27 |
| notmyname | I have a patch that doesn't seem to have been picked up by zuul: https://review.openstack.org/#/c/90380/ | 22:27 |
| fungi | mgagne: i'm going to start referring to them as "new screen" and "screen classic" | 22:28 |
| *** prad has quit IRC | 22:28 | |
| fungi | notmyname: saw the same on another change from that vintage earlier. add a recheck no bug comment | 22:28 |
| notmyname | fungi: ok, thanks | 22:29 |
| notmyname | fungi: that did it. thanks | 22:29 |
| *** etoews_ has quit IRC | 22:29 | |
| fungi | notmyname: one of our restarts interrupted things between when jenkins cleared its vote and would have reported, so it has no verify vote at all (causing the approval trigger not to fire) | 22:29 |
| *** zigo_ has joined #openstack-infra | 22:35 | |
| *** persia__ has joined #openstack-infra | 22:35 | |
| *** markvoelker has quit IRC | 22:37 | |
| *** mriedem has quit IRC | 22:37 | |
| *** bhuvan has quit IRC | 22:37 | |
| *** zigo has quit IRC | 22:37 | |
| *** greghaynes has quit IRC | 22:37 | |
| *** andreykurilin_ has joined #openstack-infra | 22:37 | |
| *** jgrimm has quit IRC | 22:37 | |
| *** nati_ueno has joined #openstack-infra | 22:37 | |
| *** bhuvan has joined #openstack-infra | 22:37 | |
| *** mriedem has joined #openstack-infra | 22:37 | |
| *** mark0n has quit IRC | 22:37 | |
| openstackgerrit | Marc Abramowitz proposed a change to openstack-infra/jenkins-job-builder: Pass argv to main https://review.openstack.org/90542 | 22:37 |
| *** changbl has quit IRC | 22:37 | |
| *** nati_ueno is now known as 7JTAAM1ZL | 22:37 | |
| *** nati_ueno has joined #openstack-infra | 22:37 | |
| *** eharney has quit IRC | 22:37 | |
| *** _dguerri has quit IRC | 22:37 | |
| *** bhuvan has quit IRC | 22:37 | |
| *** bhuvan has joined #openstack-infra | 22:37 | |
| jeremyb | fifieldt: i think you were looking for someone else | 22:37 |
| *** persia has quit IRC | 22:37 | |
| *** thedodd has quit IRC | 22:37 | |
| *** greghaynes_ has joined #openstack-infra | 22:37 | |
| *** proffalken has quit IRC | 22:37 | |
| *** Shrews has quit IRC | 22:37 | |
| *** 7JTAAM1ZL has quit IRC | 22:38 | |
| fungi | jeremyb: fifieldt: maybe me? | 22:38 |
| *** kiall has quit IRC | 22:38 | |
| *** doug-fish has left #openstack-infra | 22:38 | |
| jeremyb | fungi: ohai jeremys :) | 22:38 |
| fungi | (fifieldt was /msg'ing me earlier, though i expect he's asleep now) | 22:39 |
| fungi | ohai other jeremy ;) | 22:39 |
| *** BadCub has quit IRC | 22:39 | |
| jeremyb | fungi: 29 09:45:22 < fifieldt> jebl air, mor dred, jeremyb - had a spammer in #openstack, probably needs a ban | 22:39 |
| fungi | yup | 22:40 |
| fungi | there are far too many of us, which is why i'm fungi instead | 22:40 |
| jeremyb | i find the jeremy space isn't too bad. but too many jonathan, david, michael, rachel | 22:40 |
| *** mriedem has quit IRC | 22:43 | |
| *** e0ne has quit IRC | 22:46 | |
| *** e0ne has joined #openstack-infra | 22:46 | |
| *** mriedem has joined #openstack-infra | 22:57 | |
| *** flaper87 is now known as flaper87|afk | 22:57 | |
| adam_g | whats the easiest way to get results of a specific jenkins job in aggregate across multiple project check pipelines? | 22:58 |
| *** signed8bit has quit IRC | 22:58 | |
| jhesketh_ | Morning | 22:59 |
| *** david-lyle has quit IRC | 22:59 | |
| *** greghaynes_ is now known as greghaynes | 22:59 | |
| *** jhesketh_ is now known as jhesketh | 22:59 | |
| fungi | adam_g: can you be a little more specific about what it is you're wanting? a better use case description might help | 23:00 |
| *** mwagner_lap has joined #openstack-infra | 23:01 | |
| *** etoews has joined #openstack-infra | 23:02 | |
| adam_g | fungi, pass/fail over the last N hours for job $foo, which is configured to run as part of the check pipeline for multiple projects | 23:02 |
| fungi | adam_g: aha, you want statistics on job result states | 23:03 |
| fungi | adam_g: have you had a poke around in http://graphite.openstack.org/ at all? | 23:03 |
| adam_g | fungi, not yet. was wondering where the best place to start was, will take a look there now | 23:04 |
| *** e0ne has quit IRC | 23:05 | |
| *** e0ne has joined #openstack-infra | 23:06 | |
| *** yamahata has quit IRC | 23:07 | |
| *** markmc has quit IRC | 23:07 | |
| *** e0ne has quit IRC | 23:08 | |
| fungi | adam_g: http://graphite.openstack.org/render/?width=586&height=308&_salt=1398812938.615&target=stats_counts.zuul.pipeline.check.job.check-tempest-dsvm-full.SUCCESS&target=stats_counts.zuul.pipeline.check.job.check-tempest-dsvm-full.FAILURE | 23:09 |
| fungi | adam_g: something like that? | 23:09 |
| *** derekh has joined #openstack-infra | 23:10 | |
| adam_g | fungi, yeah. reading up on graphite now :) | 23:12 |
| *** dkehn is now known as dkehn_afk | 23:14 | |
| openstackgerrit | Derek Higgins proposed a change to openstack-infra/config: Skip running update-grub if it doesn't exist https://review.openstack.org/91188 | 23:14 |
| derekh | fungi: thanks for merging the F20 nodes, looks like they are failing to build at the very end in restrict_memory,sh | 23:14 |
| derekh | fungi: so how would you feel about skipping it until I find a proper alternative ? ^ | 23:15 |
| fungi | derekh: yeah, i see where it's mainly failing trying to run update-grub in restrict_memory.sh | 23:16 |
| fungi | derekh: ideas on how to adjust the kernel command line in f20? | 23:16 |
| *** andreykurilin_ has quit IRC | 23:17 | |
| fungi | derekh: if you can just work out how to pass mem=8G to the kernel at boot, we can solve it the right way | 23:17 |
| derekh | fungi: ok, I'm trying to figure out how that image got the legacy grub config file in the first place, the one I am testing on has /boot/grub/grub.conf | 23:18 |
| fungi | derekh: hmm... perhaps it has both? | 23:18 |
| *** alexpilotti has joined #openstack-infra | 23:19 | |
| fungi | an old (but unused) /etc/default/grub file and also a /boot/grub/grub.conf? | 23:19 |
| fungi | if so, then maybe merely swapping the order in that conditional would suffice | 23:19 |
| fungi | i assume /etc/default/grub is merely being used here to directly modify grub 1.x /boot/grub/menu.lst files | 23:20 |
| derekh | fungi: perhaps, its not present on the default fedora cloud image, I'll see if I can figure out where it is coming from | 23:21 |
| derekh | fungi: hopy will have a new patch in a bit | 23:21 |
| derekh | *hopefully | 23:21 |
| fungi | derekh: okay, cool. i'm semi-around for a few more hours to give it a spin if so | 23:22 |
| derekh | fungi: ok, thanks | 23:22 |
| *** bknudson has quit IRC | 23:23 | |
| *** melwitt has quit IRC | 23:23 | |
| *** zns has quit IRC | 23:23 | |
| *** manishg has quit IRC | 23:25 | |
| openstackgerrit | James E. Blair proposed a change to stackforge/gertty: Initial commit https://review.openstack.org/91191 | 23:27 |
| * jeblair is amused to note that gertty is unabled to display the diff at all for thet because of https://storyboard.openstack.org/#!/story/70 | 23:29 | |
| jeblair | mordred, fungi: https://review.openstack.org/gitweb?p=stackforge/gertty.git;a=blob;f=project.config;hb=36b9e169bc6451baaf1ce57983c199e421d7363d | 23:30 |
| *** hogepodge has quit IRC | 23:30 | |
| jeblair | mordred, fungi: i don't think m-p did the right thing there | 23:30 |
| jeblair | infra-manual worked, however | 23:31 |
| *** ArxCruz has quit IRC | 23:31 | |
| jeblair | GithubException: 403 {u'documentation_url': u'https://developer.github.com/v3/#rate-limiting', u'message': u'API rate limit exceeded for openstack-project-creator.'} | 23:31 |
| *** ArxCruz_ has joined #openstack-infra | 23:31 | |
| jeblair | also ERROR:manage_projects:Failed to add groups file for project: stackforge/gertty | 23:32 |
| jeblair | mordred: that seems really really familiar ^ | 23:32 |
| harlowja | jeblair for https://github.com/stackforge/gertty have u tried https://pypi.python.org/pypi/gerrit-view :) | 23:34 |
| harlowja | i saw gertty pop up :-P | 23:36 |
| harlowja | https://github.com/harlowja/gerrit_view/#cgerrit | 23:36 |
| jeblair | harlowja: gertty is to be a complete replacement for the gerrit ui, that supports an email/news reading like workflow, which i think is a little different from cgerrit... however -- | 23:36 |
| *** amcrn has quit IRC | 23:36 | |
| harlowja | hmmm | 23:36 |
| *** mrodden has quit IRC | 23:36 | |
| jeblair | harlowja: i was thinking of you when i started it, and it uses urwid. | 23:36 |
| harlowja | lol | 23:37 |
| harlowja | cool | 23:37 |
| jeblair | harlowja: i'm hoping you'll find it interesting. :) | 23:37 |
| harlowja | could just suck over cgerrit, i'm fine with that :-P | 23:37 |
| harlowja | then have 1 repo for all | 23:37 |
| harlowja | or just all of https://pypi.python.org/pypi/gerrit-view | 23:37 |
| harlowja | i'm fine with that to :) | 23:38 |
| harlowja | u have my permission ;) | 23:38 |
| *** matsuhashi has joined #openstack-infra | 23:38 | |
| *** sweston has quit IRC | 23:39 | |
| *** wenlock has quit IRC | 23:39 | |
| jeblair | harlowja: cool; worth thinking about -- it's my first urwid program, and i had a lot of things i needed to explore from scratch to start with (the new rest api, offline syncing, and just learning to use urwid) | 23:39 |
| harlowja | jeblair ya, urwid is a tough one to figure out :) | 23:39 |
| harlowja | especially customizing it, lol | 23:39 |
| jeblair | tell me about it :) i'm just now understanding how to make not-insane reusable widgets | 23:40 |
| harlowja | lol | 23:40 |
| harlowja | ya, i'm not still that good at it :-P | 23:40 |
| jeblair | harlowja: but that commit has all the fundamentals working | 23:40 |
| jeblair | if i can figure out the manage-projects issue, i should be able to land it with gertty | 23:41 |
| harlowja | :) | 23:41 |
| openstackgerrit | Khai Do proposed a change to openstack-infra/config: deploy gerrit plugins into folder with plugin name https://review.openstack.org/91192 | 23:41 |
| openstackgerrit | Khai Do proposed a change to openstack-infra/config: puppetize installation of gerrit third party plugins https://review.openstack.org/91193 | 23:41 |
| zaro | jeblair: ^ | 23:41 |
| jeblair | zaro: cool, thanks | 23:41 |
| harlowja | jeblair checking it out | 23:41 |
| harlowja | will try to instill what i learned about urwid :-P | 23:42 |
| jeblair | harlowja: that would be awesome | 23:42 |
| *** jooools has quit IRC | 23:43 | |
| jeblair | DEBUG:manage_projects:Executing command: git --git-dir=/opt/lib/jeepyb/stackforge/gertty/.git --work-tree=/opt/lib/jeepyb/stackforge/gertty checkout -B master origin/master | 23:43 |
| jeblair | INFO:manage_projects:Command said: fatal: git checkout: updating paths is incompatible with switching branches. | 23:43 |
| jeblair | mordred: ^ | 23:43 |
| *** etoews has quit IRC | 23:44 | |
| *** kiall has joined #openstack-infra | 23:47 | |
| *** pdmars has quit IRC | 23:50 | |
| jeblair | hrm, /home/gerrit2/acls/stackforge/gertty.config is empty | 23:50 |
| *** mrodden has joined #openstack-infra | 23:50 | |
| openstackgerrit | Michael Krotscheck proposed a change to openstack-infra/storyboard-webclient: Some project ui/ux updates https://review.openstack.org/91196 | 23:51 |
| *** persia__ is now known as persia | 23:51 | |
| *** persia has quit IRC | 23:51 | |
| *** persia has joined #openstack-infra | 23:51 | |
| jeblair | fungi: ping | 23:52 |
| *** etoews has joined #openstack-infra | 23:52 | |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/config: Rename gertty ACL file to correct name https://review.openstack.org/91197 | 23:52 |
| fungi | jeblair: heya | 23:53 |
| jeblair | fungi: i'm going to self-approve that | 23:53 |
| jeblair | fungi: oh, cool you're here | 23:53 |
| *** nati_uen_ has joined #openstack-infra | 23:53 | |
| jeblair | fungi: maybe you could approve it then :) | 23:53 |
| fungi | approved, yes | 23:54 |
| fungi | that explains the odd error, for sure | 23:55 |
| openstackgerrit | Khai Do proposed a change to openstack-infra/config: deploy gerrit plugins into folder with plugin name https://review.openstack.org/91192 | 23:55 |
| openstackgerrit | Khai Do proposed a change to openstack-infra/config: puppetize installation of gerrit third party plugins https://review.openstack.org/91193 | 23:55 |
| *** dguerri has joined #openstack-infra | 23:57 | |
| *** nati_ueno has quit IRC | 23:57 | |
| *** proffalken has joined #openstack-infra | 23:59 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!
