| mordred | lifeless: the manifests dir is dropped from class paths | 00:00 |
|---|---|---|
| mordred | lifeless: so modules/foo/manifests/init.pp would probably contain the foo class | 00:00 |
| mordred | lifeless: files in templates are available to be installed on the machine after being passed through erb | 00:00 |
| mordred | with every varialbe in the referencing scope injected into the scope of the template | 00:00 |
| mordred | to see an example of that - git grep 'template\(' | 00:01 |
| mordred | for instance, in modules/gerrit/manifests/init.pp - you'll see content => template('gerrit/replication.config.erb'), | 00:01 |
| mordred | which means that the results of expanding modules/gerrit/templates/replication.config.erb via erb should be placed at the location at the top of the stanze | 00:02 |
| mordred | lifeless: resources are indicated by a type, then a name, then parameters | 00:02 |
| mordred | lifeless: so file { '/home/gerrit2/review_site/etc/replication.config': content => template('gerrit/replication.config.erb'), } is declaring a file resource with the name /home/gerrit2/review_site/etc/replication.config | 00:03 |
| mordred | in any given DAG, there can only be one instantiation of a resource of the same name | 00:03 |
| mordred | many resources tend to use their name parameter as the default value of the most interesting parameter | 00:04 |
| mordred | in the case of file, $name will be the default for path => | 00:04 |
| mordred | the rest is just cargo culting existing examples and iterating until it works | 00:05 |
| mordred | :) | 00:05 |
| lifeless | ok | 00:05 |
| lifeless | thank you! | 00:05 |
| lifeless | fungi: if you're offering to draft it, that would be awesome. I'll read through and make sure I understand afterwards, so I can actually help in future. | 00:06 |
| fungi | lifeless: sure, i'll be happy to here in a bit. i'm excited to see us get rsync working there for the various reasons discussed | 00:06 |
| lifeless | fungi: I'll put up a full set of whats-needed in etherpad - https://etherpad.openstack.org/rsync | 00:08 |
| mordred | what you don't fully understand all of puppet from that description? | 00:10 |
| lifeless | mordred: It's enough for me to dig and follow through | 00:11 |
| lifeless | -very- helpful | 00:11 |
| lifeless | doesn't mean I've internalised it yet | 00:11 |
| mordred | ++ | 00:12 |
| *** nijaba has quit IRC | 00:16 | |
| *** nijaba has joined #openstack-infra | 00:17 | |
| *** nijaba has quit IRC | 00:17 | |
| *** nijaba has joined #openstack-infra | 00:17 | |
| *** sarob has joined #openstack-infra | 00:17 | |
| mordred | fungi: btw - my friends band, Teen Girl Scientist Monthly, is playing the Cave in Chapel Hill tonight at 10 | 00:19 |
| mordred | they're one of the ones I work with up here | 00:19 |
| mordred | Shrews: ^^ | 00:19 |
| *** sarob has quit IRC | 00:22 | |
| Shrews | mordred: ugh, picking up some friends at the airport at 11. would've liked to have seen them. | 00:23 |
| *** CaptTofu has quit IRC | 00:28 | |
| *** CaptTofu has joined #openstack-infra | 00:29 | |
| fungi | mordred: oh, wow. too bad we're old fuddy-duddies and turning in early | 00:30 |
| * fungi should have planned ahead | 00:30 | |
| Shrews | mordred: where are they tomorrow? | 00:42 |
| *** CaptTofu has quit IRC | 00:53 | |
| *** CaptTofu has joined #openstack-infra | 00:53 | |
| *** pabelanger has joined #openstack-infra | 01:01 | |
| *** nijaba has quit IRC | 01:16 | |
| *** nijaba has joined #openstack-infra | 01:17 | |
| *** nijaba has quit IRC | 01:17 | |
| *** nijaba has joined #openstack-infra | 01:17 | |
| *** sarob has joined #openstack-infra | 01:19 | |
| *** sarob has quit IRC | 01:23 | |
| *** yaguang has joined #openstack-infra | 01:40 | |
| *** sarob has joined #openstack-infra | 01:50 | |
| lifeless | fungi: o/ | 01:51 |
| *** yaguang has quit IRC | 01:52 | |
| fungi | lifeless: working on it now, but on the wearable so my typing's a little slow | 01:52 |
| lifeless | fungi: oh cool! just hadn't heard anything :> | 01:52 |
| fungi | gradually getting faster on this 7-key chording keyboard ;) | 01:53 |
| *** yaguang has joined #openstack-infra | 01:53 | |
| *** sarob has quit IRC | 01:54 | |
| lifeless | do you have a HUD? | 01:55 |
| *** CaptTofu has quit IRC | 02:01 | |
| *** CaptTofu has joined #openstack-infra | 02:02 | |
| openstackgerrit | Jeremy Stanley proposed a change to openstack-infra/config: Add a read-only rsync server to the pypi mirror https://review.openstack.org/40140 | 02:03 |
| fungi | lifeless: ^ | 02:03 |
| lifeless | wicked | 02:03 |
| fungi | lifeless: yeah, it's a converted pair of dvd viewer goggles | 02:04 |
| fungi | only 640x480 on ntsc, but usable | 02:05 |
| fungi | there are finally hdmi 720p res models on the market this year at reasonable prices, so i'll likely upgrade soon | 02:07 |
| lifeless | fungi: I so want to play with that in HK, if possible. | 02:07 |
| fungi | not really planning to risk bringing diy electronics through customs this time around | 02:08 |
| lifeless | ah, fair enough | 02:09 |
| lifeless | sad state of affairs | 02:09 |
| fungi | the usa is far too paranoid, i'd mostly worry about getting detained on re-entry | 02:10 |
| fungi | but next domestic trip where i think i might bump into you, i'll be sure to bring it along | 02:11 |
| lifeless | cool | 02:11 |
| fungi | by then it'll probably be smaller and better all 'round | 02:12 |
| fungi | i've got a ton of general improvements in the works for it | 02:13 |
| *** nijaba has quit IRC | 02:16 | |
| *** nijaba has joined #openstack-infra | 02:17 | |
| *** nijaba has quit IRC | 02:17 | |
| *** nijaba has joined #openstack-infra | 02:17 | |
| *** woodspa has quit IRC | 02:19 | |
| *** sarob has joined #openstack-infra | 02:20 | |
| *** sarob has quit IRC | 02:25 | |
| *** pabelanger has quit IRC | 02:30 | |
| clarkb | fungi: when are you in seattle? I think lifeless is planning a trip here too (but I think you will miss each other) | 02:35 |
| fungi | clarkb: earlier, yeah. we're there for a week starting the 17th of this month, then at sea for a week | 02:39 |
| *** pabelanger has joined #openstack-infra | 02:42 | |
| fungi | talking about making reservations for brunch at the space needle on the 18th if anybody wants to go (i expect the locals consider that to be on par with recreational oral surgery, but christine wants to do cheesy tourist things) | 02:46 |
| lifeless | 16-20 sept for me | 02:48 |
| *** yaguang has quit IRC | 02:49 | |
| fungi | yeah, so i'm missing you by a month | 02:49 |
| *** sarob has joined #openstack-infra | 02:51 | |
| lifeless | where are you based? | 02:53 |
| fungi | opposite end of the rock, raleigh, north carolina, several thousand miles from seattle | 02:54 |
| *** sarob has quit IRC | 02:55 | |
| lifeless | that makes it what 2200 for you? | 02:55 |
| fungi | for people who abide by local time here, it's 22:56 | 02:56 |
| fungi | aka "eastern daylight time" | 02:57 |
| lifeless | kk | 02:59 |
| *** yaguang has joined #openstack-infra | 03:01 | |
| *** esp has joined #openstack-infra | 03:06 | |
| *** yaguang has quit IRC | 03:06 | |
| *** esp has quit IRC | 03:07 | |
| *** yaguang has joined #openstack-infra | 03:08 | |
| *** dkliban has joined #openstack-infra | 03:13 | |
| *** nijaba has quit IRC | 03:16 | |
| *** HenryG has quit IRC | 03:17 | |
| *** nijaba has joined #openstack-infra | 03:18 | |
| *** nijaba has quit IRC | 03:18 | |
| *** nijaba has joined #openstack-infra | 03:18 | |
| *** sarob has joined #openstack-infra | 03:21 | |
| *** sarob has quit IRC | 03:25 | |
| *** SergeyLukjanov has joined #openstack-infra | 03:39 | |
| *** bnemec_away is now known as bnemec | 03:43 | |
| *** sarob has joined #openstack-infra | 03:52 | |
| *** sarob has quit IRC | 03:56 | |
| *** pabelanger has quit IRC | 03:58 | |
| *** pabelanger has joined #openstack-infra | 04:14 | |
| *** niska has quit IRC | 04:15 | |
| *** nijaba has quit IRC | 04:17 | |
| *** nijaba has joined #openstack-infra | 04:18 | |
| hub_cap | mordred: thx for merging the python-netaddr stuffs in devstack | 04:22 |
| *** sarob has joined #openstack-infra | 04:22 | |
| *** sarob has quit IRC | 04:27 | |
| *** niska has joined #openstack-infra | 04:39 | |
| hub_cap | our issues look to be fixed at present mordred clarkb etc | 04:42 |
| *** emagana has joined #openstack-infra | 04:44 | |
| *** boris-42 has joined #openstack-infra | 04:46 | |
| *** sarob has joined #openstack-infra | 04:53 | |
| lifeless | Hi; what blockers are there to landing https://review.openstack.org/#/c/39858/ ? in -config ? | 04:55 |
| lifeless | other than no infra folk in my tz ? | 04:55 |
| lifeless | we're blocked on landing the packaging stuff until it's in. | 04:55 |
| *** sarob has quit IRC | 04:57 | |
| *** dguitarbite has joined #openstack-infra | 05:01 | |
| *** marun has joined #openstack-infra | 05:05 | |
| dstufft | lifeless: just noticed you called me out in an email, checking now | 05:08 |
| dstufft | lifeless: fwiw I mostly ignore gerrit email because there's a metric ton of it | 05:09 |
| lifeless | dstufft: there is that | 05:15 |
| lifeless | dstufft: fedora folk having pip install -U distribute uninstall setuptools | 05:15 |
| *** pabelanger has quit IRC | 05:16 | |
| *** afazekas_ has joined #openstack-infra | 05:17 | |
| *** nijaba has quit IRC | 05:17 | |
| *** nijaba has joined #openstack-infra | 05:17 | |
| *** nicedice has quit IRC | 05:20 | |
| *** dina_belova has joined #openstack-infra | 05:22 | |
| *** sarob has joined #openstack-infra | 05:24 | |
| *** dina_belova has quit IRC | 05:25 | |
| *** sarob has quit IRC | 05:28 | |
| *** SergeyLukjanov has quit IRC | 05:40 | |
| *** emagana_ has joined #openstack-infra | 05:41 | |
| *** emagana has quit IRC | 05:44 | |
| *** sarob has joined #openstack-infra | 05:55 | |
| *** sarob has quit IRC | 06:00 | |
| *** emagana_ has quit IRC | 06:00 | |
| *** amotoki has joined #openstack-infra | 06:10 | |
| *** nijaba has quit IRC | 06:18 | |
| *** nijaba has joined #openstack-infra | 06:18 | |
| *** nijaba has quit IRC | 06:18 | |
| *** nijaba has joined #openstack-infra | 06:18 | |
| *** sarob has joined #openstack-infra | 06:25 | |
| *** odyssey4me has joined #openstack-infra | 06:28 | |
| *** sarob has quit IRC | 06:30 | |
| *** tian has joined #openstack-infra | 06:32 | |
| *** dguitarbite has quit IRC | 06:36 | |
| *** odyssey4me has quit IRC | 06:46 | |
| *** odyssey4me has joined #openstack-infra | 06:53 | |
| *** sarob has joined #openstack-infra | 06:56 | |
| *** nijaba has quit IRC | 06:59 | |
| *** sarob has quit IRC | 07:00 | |
| *** boris-42 has quit IRC | 07:07 | |
| *** nijaba has joined #openstack-infra | 07:08 | |
| *** nijaba has quit IRC | 07:08 | |
| *** nijaba has joined #openstack-infra | 07:08 | |
| *** thomasbiege has joined #openstack-infra | 07:08 | |
| *** Vivek has quit IRC | 07:15 | |
| *** xchu has joined #openstack-infra | 07:17 | |
| *** Vivek has joined #openstack-infra | 07:17 | |
| *** Vivek is now known as Guest58376 | 07:17 | |
| *** thomasbiege has quit IRC | 07:19 | |
| *** sarob has joined #openstack-infra | 07:26 | |
| *** psedlak has joined #openstack-infra | 07:31 | |
| *** sarob has quit IRC | 07:31 | |
| *** psedlak has quit IRC | 07:35 | |
| *** psedlak has joined #openstack-infra | 07:35 | |
| *** fbo_away is now known as fbo | 07:39 | |
| *** xchu has quit IRC | 07:43 | |
| *** sarob has joined #openstack-infra | 07:57 | |
| *** sarob has quit IRC | 08:01 | |
| *** boris-42 has joined #openstack-infra | 08:07 | |
| *** afazekas_ is now known as afazekas_food | 08:07 | |
| *** BobBall_Away is now known as BobBall | 08:08 | |
| *** xchu has joined #openstack-infra | 08:08 | |
| *** nijaba has quit IRC | 08:08 | |
| *** nijaba has joined #openstack-infra | 08:09 | |
| *** nijaba has quit IRC | 08:09 | |
| *** nijaba has joined #openstack-infra | 08:09 | |
| *** sarob has joined #openstack-infra | 08:27 | |
| *** sarob has quit IRC | 08:32 | |
| *** __cyril__ has joined #openstack-infra | 08:40 | |
| *** ruhe has joined #openstack-infra | 08:50 | |
| *** sarob has joined #openstack-infra | 08:58 | |
| *** sarob has quit IRC | 09:02 | |
| *** Ryan_Lane has quit IRC | 09:05 | |
| *** nijaba has quit IRC | 09:08 | |
| *** nijaba_ has joined #openstack-infra | 09:08 | |
| *** lifeless has quit IRC | 09:10 | |
| *** fbo is now known as fbo_away | 09:11 | |
| *** afazekas_ has joined #openstack-infra | 09:13 | |
| *** fbo_away is now known as fbo | 09:14 | |
| *** lifeless has joined #openstack-infra | 09:20 | |
| *** dkehn_ has joined #openstack-infra | 09:22 | |
| *** dkehn has quit IRC | 09:23 | |
| *** sarob has joined #openstack-infra | 09:28 | |
| *** sarob has quit IRC | 09:32 | |
| *** dkliban has quit IRC | 09:34 | |
| *** dina_belova has joined #openstack-infra | 09:37 | |
| *** SergeyLukjanov has joined #openstack-infra | 09:38 | |
| *** sarob has joined #openstack-infra | 09:59 | |
| *** sarob has quit IRC | 10:04 | |
| *** nijaba has joined #openstack-infra | 10:08 | |
| *** nijaba has quit IRC | 10:08 | |
| *** nijaba has joined #openstack-infra | 10:08 | |
| *** nijaba_ has quit IRC | 10:08 | |
| *** xchu has quit IRC | 10:09 | |
| *** dina_belova has quit IRC | 10:11 | |
| *** ruhe has quit IRC | 10:15 | |
| *** NobodyCam has quit IRC | 10:25 | |
| *** devananda has quit IRC | 10:28 | |
| *** sarob has joined #openstack-infra | 10:29 | |
| *** yaguang has quit IRC | 10:34 | |
| *** sarob has quit IRC | 10:34 | |
| *** ruhe has joined #openstack-infra | 10:43 | |
| *** boris-42_ has joined #openstack-infra | 10:44 | |
| *** boris-42 has quit IRC | 10:45 | |
| *** jhesketh_ has quit IRC | 10:45 | |
| *** boris-42_ is now known as boris-42 | 10:45 | |
| *** jhesketh_ has joined #openstack-infra | 10:46 | |
| *** dina_belova has joined #openstack-infra | 11:02 | |
| *** nijaba has quit IRC | 11:08 | |
| *** nijaba has joined #openstack-infra | 11:08 | |
| *** nijaba has quit IRC | 11:08 | |
| *** nijaba has joined #openstack-infra | 11:08 | |
| *** jhesketh_ has quit IRC | 11:22 | |
| *** sarob has joined #openstack-infra | 11:30 | |
| *** woodspa has joined #openstack-infra | 11:34 | |
| *** sarob has quit IRC | 11:35 | |
| *** lcestari has joined #openstack-infra | 11:46 | |
| *** jhesketh_ has joined #openstack-infra | 11:48 | |
| openstackgerrit | Sascha Peilicke proposed a change to openstack/requirements: Drop Cheetah global requirement https://review.openstack.org/40206 | 11:50 |
| *** weshay has joined #openstack-infra | 11:56 | |
| *** NobodyCam has joined #openstack-infra | 11:57 | |
| *** devananda has joined #openstack-infra | 11:59 | |
| *** dina_belova has quit IRC | 11:59 | |
| *** sarob has joined #openstack-infra | 12:01 | |
| *** sandywalsh has quit IRC | 12:02 | |
| *** beagles has joined #openstack-infra | 12:03 | |
| *** sarob has quit IRC | 12:06 | |
| *** nijaba_ has joined #openstack-infra | 12:08 | |
| *** dina_belova has joined #openstack-infra | 12:09 | |
| *** nijaba has quit IRC | 12:09 | |
| *** sandywalsh has joined #openstack-infra | 12:15 | |
| *** rfolco has joined #openstack-infra | 12:15 | |
| *** thomasem has joined #openstack-infra | 12:20 | |
| *** CaptTofu has quit IRC | 12:22 | |
| *** CaptTofu has joined #openstack-infra | 12:22 | |
| *** HenryG has joined #openstack-infra | 12:25 | |
| *** ruhe has quit IRC | 12:27 | |
| *** ruhe has joined #openstack-infra | 12:27 | |
| *** adalbas has quit IRC | 12:28 | |
| *** chrisstreeter has joined #openstack-infra | 12:32 | |
| *** chrisstreeter has quit IRC | 12:35 | |
| *** krtaylor has quit IRC | 12:40 | |
| *** chrisstreeter has joined #openstack-infra | 12:40 | |
| *** chrisstreeter has quit IRC | 12:41 | |
| openstackgerrit | Thierry Carrez proposed a change to openstack-infra/odsreg: Allow multiple allocations for a topic https://review.openstack.org/40212 | 12:43 |
| *** chuckieb has joined #openstack-infra | 12:44 | |
| *** adalbas has joined #openstack-infra | 12:44 | |
| openstackgerrit | David Caro proposed a change to openstack-infra/jenkins-job-builder: Added matrix trigger option to email-ext publisher https://review.openstack.org/40214 | 12:49 |
| *** xchu has joined #openstack-infra | 13:01 | |
| *** sarob has joined #openstack-infra | 13:02 | |
| *** sarob has quit IRC | 13:06 | |
| *** nijaba has joined #openstack-infra | 13:09 | |
| *** nijaba_ has quit IRC | 13:09 | |
| *** dkehn_ is now known as dkehn | 13:10 | |
| *** ruhe has quit IRC | 13:18 | |
| *** dprince has joined #openstack-infra | 13:18 | |
| *** dina_belova has quit IRC | 13:20 | |
| *** dina_belova has joined #openstack-infra | 13:21 | |
| *** ruhe has joined #openstack-infra | 13:21 | |
| *** pentameter has joined #openstack-infra | 13:24 | |
| *** mriedem has joined #openstack-infra | 13:24 | |
| *** dina_belova has quit IRC | 13:25 | |
| *** dina_belova has joined #openstack-infra | 13:25 | |
| *** pabelanger has joined #openstack-infra | 13:26 | |
| *** dina_belova has quit IRC | 13:30 | |
| *** kirankv has joined #openstack-infra | 13:32 | |
| *** krtaylor has joined #openstack-infra | 13:32 | |
| *** ruhe has quit IRC | 13:35 | |
| *** changbl has quit IRC | 13:35 | |
| *** jpeeler has quit IRC | 13:37 | |
| *** jpeeler has joined #openstack-infra | 13:38 | |
| *** jpeeler has joined #openstack-infra | 13:38 | |
| *** pabelanger has quit IRC | 13:39 | |
| sdague | any coremudgeons around that could hold a vm? I think the fastest way to get to the bottom of this grenade issue is to poke around on it it after the fail | 13:44 |
| *** ruhe has joined #openstack-infra | 13:44 | |
| fungi | sdague: how frequent are they? that will determine what tack i take with trying to catch one | 13:44 |
| sdague | fungi: it's one job | 13:44 |
| sdague | it's the proposed change to get us to updating from requirements | 13:45 |
| sdague | so I can trigger the job that will fail it now | 13:45 |
| fungi | ah, simple | 13:45 |
| fungi | i may be able to reproduce without holding one, just following my test simulation instructions in the d-g readme | 13:45 |
| sdague | fungi: https://jenkins.openstack.org/job/gate-grenade-devstack-vm/24561/ | 13:46 |
| sdague | if you hold that vm | 13:46 |
| sdague | you've got about 30 minutes before it finishes | 13:46 |
| sdague | but it *will fail* | 13:46 |
| fungi | can do | 13:46 |
| sdague | thanks much | 13:47 |
| *** dina_belova has joined #openstack-infra | 13:48 | |
| *** vijendar has joined #openstack-infra | 13:49 | |
| *** fifieldt has quit IRC | 13:54 | |
| *** _TheDodd_ has joined #openstack-infra | 13:56 | |
| fungi | sdague: 15.185.189.23 is the slave. it's marked held and i've added the rsa ssh key from your lp profile to the jenkins user on it | 13:56 |
| *** prad_ has joined #openstack-infra | 13:57 | |
| *** ruhe has quit IRC | 14:00 | |
| *** blamar has quit IRC | 14:00 | |
| sdague | fungi: cool | 14:01 |
| sdague | thanks | 14:01 |
| sdague | man, you guys are sensible and use your class A for public facing stuff :) | 14:01 |
| *** adalbas has quit IRC | 14:02 | |
| *** sarob has joined #openstack-infra | 14:04 | |
| *** odyssey4me has quit IRC | 14:04 | |
| *** dina_belova has quit IRC | 14:04 | |
| *** sarob has quit IRC | 14:08 | |
| *** wu_wenxiang has joined #openstack-infra | 14:09 | |
| *** nijaba_ has joined #openstack-infra | 14:09 | |
| *** nijaba has quit IRC | 14:10 | |
| *** blamar has joined #openstack-infra | 14:10 | |
| *** fbo is now known as fbo_away | 14:12 | |
| *** afazekas_ has quit IRC | 14:13 | |
| *** rnirmal has joined #openstack-infra | 14:17 | |
| *** dina_belova has joined #openstack-infra | 14:17 | |
| *** dina_belova has quit IRC | 14:18 | |
| *** adalbas has joined #openstack-infra | 14:18 | |
| fungi | who's "you guys"? the hpcloud peeps i guess | 14:18 |
| *** dina_belova has joined #openstack-infra | 14:20 | |
| fungi | grr... sdague: sorry, looks like that vm got deleted | 14:20 |
| fungi | i think the recent d-g refactoring no longer notices if a vm is set to hold state when it marks for deletion | 14:20 |
| *** fbo_away is now known as fbo | 14:21 | |
| *** dkranz has quit IRC | 14:21 | |
| fungi | sdague: i'm manually launching an identical vm right now where i'll fire up the same test and get you ssh access to it, then i'll track down where we broke the hold state stuff in d-g | 14:24 |
| *** datsun180b has joined #openstack-infra | 14:25 | |
| *** burt has joined #openstack-infra | 14:26 | |
| *** ruhe has joined #openstack-infra | 14:27 | |
| *** vogxn has joined #openstack-infra | 14:30 | |
| *** dolphm has joined #openstack-infra | 14:31 | |
| sdague | fungi: ok, cool | 14:32 |
| sdague | fungi: sorry "you guys" == HP | 14:32 |
| * fungi is not hp, but will gladly accept this award on their behalf | 14:33 | |
| *** pcrews has joined #openstack-infra | 14:33 | |
| sdague | heh, yeh, I remembered now | 14:34 |
| *** cthulhup_ has joined #openstack-infra | 14:35 | |
| *** sarob has joined #openstack-infra | 14:35 | |
| *** cthulhup_ has quit IRC | 14:36 | |
| *** emagana has joined #openstack-infra | 14:37 | |
| sdague | fungi: ummm.... any idea why grenade-core is empty? | 14:38 |
| sdague | https://review.openstack.org/#/admin/groups/188,members | 14:38 |
| *** sarob has quit IRC | 14:39 | |
| fungi | none whatsoever | 14:39 |
| fungi | it does seem to be used in the grenade project acl, so it ought to have users | 14:39 |
| fungi | it's also not self-managed (owned by administrators instead) | 14:40 |
| fungi | who do you want in it? | 14:40 |
| fungi | also, i assume it was in use previously? i'll look to see if that acl changed recently to create a new core group | 14:41 |
| *** dkliban has joined #openstack-infra | 14:41 | |
| *** rcleere has joined #openstack-infra | 14:42 | |
| *** markmcclain has joined #openstack-infra | 14:43 | |
| fungi | sdague: the acl used to reference devstack-core instead, then you proposed https://review.openstack.org/36902 which merged about 3 weeks ago to create grenade-core | 14:44 |
| fungi | we probably just never added anyone to it after that | 14:44 |
| sdague | heh, ok | 14:44 |
| sdague | yeh, can you put dtroyer and I as +2 on it | 14:44 |
| fungi | coming right up | 14:44 |
| *** wu_wenxiang has quit IRC | 14:46 | |
| fungi | you're both in grenade-core now | 14:46 |
| sdague | awesome, thanks | 14:48 |
| *** cppcabrera has joined #openstack-infra | 14:52 | |
| *** cppcabrera has left #openstack-infra | 14:53 | |
| fungi | i've got gate-grenade-devstack-vm running on a manually-launched vm now with the same changeset and parameters used in the job you pointed me to, so i'll give you a heads up once it fails and get you the ip address so you can look around | 14:54 |
| *** Guest98707 has joined #openstack-infra | 14:54 | |
| *** Guest98707 has quit IRC | 14:55 | |
| *** CliMz has joined #openstack-infra | 14:57 | |
| *** CliMz has joined #openstack-infra | 14:58 | |
| sdague | fungi: awesome, thank you | 14:59 |
| *** emagana has quit IRC | 15:00 | |
| mordred | morning sdague ! | 15:01 |
| sdague | morning | 15:01 |
| mordred | sdague: I made some progress on some things after you left on friday - but not the grenade issue | 15:01 |
| sdague | yeh, fungi is spinning me up a vm for the grenade thing, so I can poke aroun | 15:02 |
| *** changbl has joined #openstack-infra | 15:02 | |
| *** sarob has joined #openstack-infra | 15:05 | |
| mordred | awesome | 15:06 |
| burt | mordred: thanks for getting the dependency hell cleaned up. Clean Precise + devstack works now. | 15:06 |
| mordred | burt: w00t! | 15:08 |
| mordred | burt, sdague I'm going to be an ass about persuing the larger patch, btw | 15:09 |
| *** nijaba_ has quit IRC | 15:09 | |
| mordred | sdague: I think that, with the work we're doing on global requirements, chasing the distro packages for python packages is a waste of time and counter productive | 15:10 |
| *** nijaba has joined #openstack-infra | 15:10 | |
| *** sarob has quit IRC | 15:10 | |
| mordred | sdague: although, if we had a tool such as the one you discuss on the code review, I could see that being decent | 15:10 |
| burt | mordred: sounds good. I'll be an ass if devstack breaks again for me. | 15:11 |
| mordred | :) | 15:11 |
| sdague | yeh, I think realistically packages vs. pip is actually the same problem as different pip installs in different projects | 15:11 |
| *** rnirmal_ has joined #openstack-infra | 15:12 | |
| *** rnirmal has quit IRC | 15:13 | |
| *** rnirmal_ is now known as rnirmal | 15:13 | |
| *** vogxn has quit IRC | 15:14 | |
| *** mrodden1 has quit IRC | 15:14 | |
| mordred | sdague: I just responded more long form on the review | 15:16 |
| mordred | (since probably if there is a back and forth it would be good to capture it there) | 15:16 |
| *** xchu has quit IRC | 15:17 | |
| ttx | Just looked into setting up CIVS locally so that we can run ballots through alternative algorithms to see how well they perform. | 15:18 |
| ttx | Pretty simple stuff. | 15:18 |
| mordred | ttx: neat! should we consider running our own for real? or just for testing? | 15:19 |
| *** dkranz has joined #openstack-infra | 15:19 | |
| ttx | I think letting a third-party run the voting infra is sort of a feature | 15:19 |
| sdague | mordred: sure, or bring it out to the mailing list | 15:20 |
| ttx | mordred: but yes, I almost convinced myselff to write a puppet recipe for setting it up | 15:20 |
| sdague | typically that's what I've done when there turns out to be an actual dispute in the review | 15:20 |
| *** reed has joined #openstack-infra | 15:20 | |
| ttx | mordred: it uses weird patching of Perl files with configuration values | 15:20 |
| mordred | ttx: good point re: third party running it being a feature | 15:21 |
| fungi | sdague: you should be able to ssh to 166.78.161.26 as jenkins now. that manual run produced the same failure as the job console log you pointed me to (note i simply followed the instructions in the d-g readme for simulating a job) | 15:21 |
| *** __cyril__ has left #openstack-infra | 15:21 | |
| sdague | fungi: thanks, I'm in | 15:22 |
| fungi | sdague: let me know when you're done with it (take your time though) so i can delete it | 15:22 |
| ttx | fungi, jeblair: Make sure to enable "Enable detailed ballot reporting" option on future elections you set up | 15:22 |
| fungi | ttx: does that provide access to the anonymized raw data for analysis? | 15:23 |
| ttx | fungi, jeblair: that way we can download the anonymized ballots and run the results through various algorithms and see how well/bad they perform | 15:23 |
| fungi | perfect | 15:23 |
| *** blamar has quit IRC | 15:23 | |
| *** amotoki has quit IRC | 15:24 | |
| ttx | fungi: we need to "test" the representative proportional algorithm in particular. | 15:24 |
| ttx | actually... | 15:24 |
| ttx | we don't even need to run our own to do that. | 15:24 |
| jeblair | fungi, clarkb, mordred: so, ah, jbryce has moved openstack.org email to something new and it can no longer be managed in the cloud sites webui. | 15:24 |
| sdague | mordred: so.... the issue seems to be that oslo.config gets installed in grizzly because it's coming from pip | 15:25 |
| *** ruhe has quit IRC | 15:25 | |
| sdague | but in havana, with global requirements, we use the git tree | 15:25 |
| ttx | we can set up a "test" consultation and upload results on the CIVS site itself. | 15:25 |
| sdague | but don't install | 15:25 |
| ttx | I guess I reproduced the install for no good reason :) | 15:25 |
| sdague | and as such nova-manage is picking up the pip installed tree and not the git tree | 15:25 |
| sdague | how do we go about addressing that? | 15:25 |
| fungi | jeblair: is that what his message meant? i had never tried to manage my o.o mailbox via anything besides imap | 15:25 |
| fungi | ttx: setting up your own provides additional insights into its inner workings at least, i would expect | 15:27 |
| sdague | do we just install oslo.* as well as doing setup.py ? | 15:27 |
| *** emagana has joined #openstack-infra | 15:27 | |
| sdague | sorry, setup.py develop | 15:27 |
| *** emagana has quit IRC | 15:27 | |
| ttx | fungi: it does :) | 15:27 |
| openstackgerrit | Monty Taylor proposed a change to openstack-dev/hacking: Synced with global requirements https://review.openstack.org/40065 | 15:27 |
| jeblair | fungi: yeah; i have asked him for an admin account, but considering the number of system-y mailboxes we manage, i think the whole root team should have either one shared admin account, or one each. | 15:28 |
| jeblair | on the plus side, that is perhaps one less thing colocated in the main rackspace account; now it's just dns and cloud sites. | 15:29 |
| fungi | jeblair: good point | 15:29 |
| fungi | i had entirely forgotten that was also going to affect the role addresses | 15:30 |
| *** ^d has joined #openstack-infra | 15:30 | |
| *** ^d has joined #openstack-infra | 15:30 | |
| *** mrodden has joined #openstack-infra | 15:30 | |
| *** mgagne has joined #openstack-infra | 15:31 | |
| *** vijendar has quit IRC | 15:31 | |
| *** CliMz has quit IRC | 15:31 | |
| *** ruhe has joined #openstack-infra | 15:32 | |
| mordred | jeblair: oh, that's great! it's wonderful that it's being managed by something other than the world-class email admin we have... | 15:32 |
| *** vogxn has joined #openstack-infra | 15:32 | |
| *** blamar has joined #openstack-infra | 15:33 | |
| *** blamar has quit IRC | 15:33 | |
| mordred | sdague: wait, didn't we add that already? | 15:33 |
| jeblair | mordred: it sounded like it's another rackspace thing, maybe a new version of their hosted email service or something. | 15:33 |
| jeblair | mordred, ttx: one of my talks has been accepted at linuxcon europe | 15:35 |
| mordred | jeblair: same here | 15:35 |
| sdague | mordred: no, we were just doing this - https://github.com/openstack-dev/devstack/blob/master/lib/oslo | 15:35 |
| * ttx skips | 15:35 | |
| * mordred looks | 15:35 | |
| sdague | mordred: so is my only issue I need to add setup.py install there? | 15:35 |
| jeblair | ttx: bummer :( | 15:35 |
| mordred | no - it should do the same thing | 15:35 |
| mordred | sdague: setup_develop should be running pip install -e now | 15:36 |
| ttx | jeblair: it's like release week +1 | 15:36 |
| sdague | mordred: pip install isn't the issue | 15:36 |
| sdague | mordred: here is the issue | 15:36 |
| sdague | in grenade | 15:36 |
| *** blamar has joined #openstack-infra | 15:36 | |
| *** sarob has joined #openstack-infra | 15:36 | |
| sdague | during grizzly install: | 15:36 |
| ttx | jeblair: aka summit week -2 | 15:36 |
| sdague | * pip install oslo.conf 1.1 | 15:36 |
| sdague | do lots of stuff | 15:36 |
| sdague | during havana upgrade: | 15:37 |
| sdague | pull oslo.config from git | 15:37 |
| jeblair | ttx: ah, your only chance to unwind. | 15:37 |
| sdague | run python setup.py develop in oslo.config tree | 15:37 |
| sdague | (but we don't install it | 15:37 |
| ttx | jeblair: it's actually summit scheduling high tide | 15:37 |
| sdague | so ... /usr/local/lib/python2.7/dist-packages/oslo | 15:37 |
| sdague | is still 1.1.1 | 15:37 |
| ttx | jeblair: not sure I will unwind at all. | 15:37 |
| sdague | not the git version | 15:37 |
| mordred | but the easy-install.pth file in /usr/local/lib/python2.7/dist-packages should have an entry | 15:38 |
| sdague | it does | 15:39 |
| sdague | apparently that's not good enough? | 15:39 |
| reed | does anybody here knows the number of downloads from tarballs on launchpad? | 15:39 |
| jeblair | ttx: ugh. well, uh, enjoy your current holiday. :) | 15:39 |
| mordred | looking | 15:39 |
| sdague | mordred: http://paste.openstack.org/show/43203/ | 15:39 |
| *** sarob has quit IRC | 15:40 | |
| mordred | sdague: that's what I expect that to look like | 15:42 |
| sdague | mordred: well with that setup, /usr/local/bin/nova-manage can't run | 15:43 |
| sdague | it's picking up bad old oslo | 15:43 |
| sdague | http://paste.openstack.org/show/43207/ | 15:43 |
| mordred | sdague: we could pip uninstall oslo.config before the setup_develpo call | 15:43 |
| sdague | sure, is that the right fix? | 15:44 |
| mordred | in lib/oslo | 15:44 |
| sdague | vs. installing oslo | 15:44 |
| mordred | tough to say - I'm connecting to that box right now - but min internet is SLOWWWWWWW | 15:44 |
| sdague | ok | 15:44 |
| mordred | gah. I have no keys on tha tmachine | 15:45 |
| mordred | sdague: on that machine, try pip uninstall oslo.config and then cd /opt/stack/new/oslo.config and pip install -e . | 15:45 |
| mordred | and see if it at least fixes it | 15:45 |
| sdague | jenkins@testserver:~$ sudo pip uninstall oslo.config | 15:46 |
| sdague | Can't uninstall 'oslo.config'. No files were found to uninstall. | 15:46 |
| mordred | nice | 15:46 |
| mordred | I love pip | 15:46 |
| *** sarob has joined #openstack-infra | 15:46 | |
| sdague | so, you're putting in the "Why OpenStack hates Python" talk for pycon, right? | 15:46 |
| sdague | because I seriously want to see your crazy on stage rant for that | 15:46 |
| mordred | sdague: I need to step out for lunch ... but I've got a good idea of how to reproduce the issue and poke at iterations when I get back | 15:47 |
| sdague | ok, cool | 15:47 |
| sdague | I'm pretty sure this is the last issue to get through | 15:47 |
| sdague | so could this be because nova-manage is now all command entry points? | 15:48 |
| sdague | ok, guess we'll go and get lunch here as well | 15:49 |
| *** rustlebee is now known as russellb | 15:50 | |
| *** sarob has quit IRC | 15:51 | |
| *** ruhe has quit IRC | 15:52 | |
| *** danger_fo_away has quit IRC | 15:54 | |
| *** kirankv has quit IRC | 15:54 | |
| chmouel | jey guys | 15:57 |
| chmouel | this job has failed https://review.openstack.org/#/c/40172/ | 15:57 |
| chmouel | on postgres vm | 15:57 |
| chmouel | but the log didn't get uploaded | 15:58 |
| chmouel | i.e: http://logs.openstack.org/72/40172/1/gate/gate-tempest-devstack-vm-postgres-full/b304251 | 15:58 |
| *** danger_fo_away has joined #openstack-infra | 15:58 | |
| *** danger_fo_away is now known as danger_fo | 15:59 | |
| NobodyCam | I have a question for -infra | 15:59 |
| jeblair | chmouel: there was a jenkins error; the devstack node was deleted before it started | 15:59 |
| jeblair | chmouel: i'm working on making that more robust; i hope to have an improvement in a couple of weeks | 16:00 |
| fungi | NobodyCam: you're in the right place then | 16:00 |
| jeblair | chmouel: it's getting a harder, but you can find the jenkins job with a little work; it's here: https://jenkins.openstack.org/job/gate-tempest-devstack-vm-postgres-full/39112/console | 16:01 |
| NobodyCam | is there a way to get the diff between 1 file in a review with out all the other base changes? (ie. not using the old version pull-down) | 16:02 |
| fungi | NobodyCam: here's the trick i use for that... | 16:02 |
| NobodyCam | :) | 16:02 |
| fungi | NobodyCam: view the current patchset's copy of that file diffed against base | 16:02 |
| * NobodyCam takes notes | 16:02 | |
| fungi | NobodyCam: then change the base patchset to the one you want to compare against | 16:03 |
| *** emagana has joined #openstack-infra | 16:03 | |
| fungi | NobodyCam: then look at which file ha a green "already seen it" checkmark over to the right | 16:03 |
| fungi | you still have to wade through previous patchsets changes in the same file, but at least you can pick it out of the mess of other changed files | 16:04 |
| fungi | er, not previous patchsets changes but other rebased changes in the same file | 16:04 |
| chmouel | jeblair: we have seen some of these errors since this morn | 16:04 |
| fungi | NobodyCam: not perfect, but helps whittle it down. it also helps that i've probably seen the intervening changes and probably remember what was in them to some extent | 16:05 |
| NobodyCam | ok ya I have a case whre patch #3 was approved kicked back with please rebase and not the patch has grown by onle line.. lol just trying to find that one line | 16:06 |
| NobodyCam | s/not/now | 16:06 |
| *** vijendar has joined #openstack-infra | 16:06 | |
| fungi | NobodyCam: you might also use git show on the old and new patchsets and then diff the results. bit of a pain, and probably introduces some additional noise, but might help | 16:07 |
| NobodyCam | :) | 16:07 |
| fungi | NobodyCam: or open up both patchsets in separate browser windows side by side. also not great but... | 16:07 |
| NobodyCam | yep... | 16:07 |
| fungi | in other words, i haven't found an ideal solution to the problem | 16:08 |
| NobodyCam | :-p | 16:08 |
| *** nijaba_ has joined #openstack-infra | 16:10 | |
| NobodyCam | thank you fungi :) | 16:10 |
| *** nijaba has quit IRC | 16:10 | |
| fungi | NobodyCam: yw. sorry i didn't have a better answer | 16:10 |
| pleia2 | good morning | 16:11 |
| NobodyCam | :) | 16:11 |
| fungi | morning pleia2 | 16:11 |
| NobodyCam | its all good | 16:11 |
| zaro | morning. | 16:11 |
| *** nicedice has joined #openstack-infra | 16:11 | |
| *** changbl has quit IRC | 16:14 | |
| fungi | morning zaro | 16:17 |
| *** SergeyLukjanov has quit IRC | 16:17 | |
| *** changbl has joined #openstack-infra | 16:20 | |
| *** dina_belova has quit IRC | 16:21 | |
| *** psedlak has quit IRC | 16:22 | |
| *** vogxn has quit IRC | 16:23 | |
| *** ruhe has joined #openstack-infra | 16:26 | |
| NobodyCam | TY fungi! separate browser windows did the trick | 16:27 |
| fungi | NobodyCam: great. if you ever stumble across a better way, i'd be thrilled to know about it ;) | 16:27 |
| NobodyCam | :) | 16:28 |
| fungi | i suspect something could be engineered diff'ing cleaned-up output from git show on both patchsets, but no idea whether that wheel's already been invented somewhere | 16:29 |
| *** boris-42 has quit IRC | 16:29 | |
| *** vogxn has joined #openstack-infra | 16:33 | |
| *** fbo is now known as fbo_away | 16:33 | |
| *** changbl has quit IRC | 16:35 | |
| clarkb | morning | 16:37 |
| fungi | morning clarkb | 16:37 |
| fungi | NobodyCam: actually, i just stumbled across one dirty solution to the problem... cherry-pick the new patchset onto the old one and the delta should in theory show that way | 16:38 |
| *** weshay has quit IRC | 16:40 | |
| zaro | clarkb: morning. | 16:43 |
| zaro | clarkb: hey, zmq plugin isn't showing up on jenkins plugin manager. i think we are missing something. | 16:43 |
| dansmith | is there a bug open for the swift port conflict thing? I was thinking this has been a known issue for a while, causing chance failures due to binding to 6xxx ports.. ? | 16:44 |
| jeblair | zaro, clarkb: i agree; i noticed that when i tried to use the new puppet resource for it. | 16:44 |
| clarkb | zaro: do we need to register it with them through some second channel? | 16:45 |
| clarkb | I haven't done anything like that | 16:45 |
| zaro | clarkb: i think we may just need to upload the pom.xml as well. | 16:45 |
| zaro | jeblair: puppet resource? which one? | 16:46 |
| *** pcrews has quit IRC | 16:46 | |
| zaro | clarkb: that would be my first guess. if that doesn't work, then it may be linked to the jenkins wiki somehow. | 16:46 |
| clarkb | zaro: there is a new puppet defined resource to install jenkins plugins | 16:48 |
| zaro | jeblair: it's available in the jenkins-ci repo: http://repo.jenkins-ci.org/repo/org/jenkins-ci/plugins/zmq-event-publisher/ | 16:49 |
| NobodyCam | fungi: Ahh yes that might do it.. I try it next time I run across that issue! | 16:49 |
| jeblair | zaro: see http://repo.jenkins-ci.org/repo/org/jenkins-ci/plugins/zmq-event-publisher/ | 16:49 |
| jeblair | oops | 16:50 |
| jeblair | zaro: see: https://review.openstack.org/#/c/40041/ | 16:50 |
| fungi | NobodyCam: i tested it niavely on a non-trivial rebase where some merge conflict resolution had been employed and got an error from git about an empty resulting commit, so it will probably work at least some of the time for that | 16:50 |
| fungi | er, naively | 16:50 |
| *** sarob has joined #openstack-infra | 16:51 | |
| *** sarob has quit IRC | 16:52 | |
| *** sarob has joined #openstack-infra | 16:52 | |
| zaro | jeblair: clarkb : i'll upload the pom.xml for the zmq plugin to see if it appears. it may take a few hours to propogate thru. i've heard it can take as long as 24 hrs. | 16:53 |
| *** sarob_ has joined #openstack-infra | 16:53 | |
| *** ruhe has quit IRC | 16:54 | |
| *** bearhands is now known as comstud | 16:54 | |
| *** psedlak has joined #openstack-infra | 16:55 | |
| *** sarob_ has quit IRC | 16:55 | |
| *** sarob_ has joined #openstack-infra | 16:56 | |
| *** sarob has quit IRC | 16:57 | |
| harlowja_ | mordred just say no to packages ?? ;) | 17:00 |
| sdague | dansmith: that's why it's still off by default in devstack | 17:01 |
| sdague | swift conflicts with X server ports | 17:01 |
| dtroyer | sdague, mordred: I just sent a wall of text re the package vs pip discussion to the ML, trying to summarize what we know and what is in https://review.openstack.org/40019. Please correct freely where necessary. | 17:04 |
| sdague | dtroyer: great | 17:04 |
| *** pabelanger has joined #openstack-infra | 17:06 | |
| jeblair | _TheDodd_: ping | 17:06 |
| pleia2 | can someone do me a favor and check to see if /home/cgit/.ssh/authorized_keys on git.o.o is populated? | 17:07 |
| pleia2 | (trying to see if I properly defined the contents variable in my puppet config) | 17:07 |
| jeblair | pleia2: will do | 17:07 |
| pleia2 | thank you | 17:08 |
| jeblair | pleia2: gerrit-code-review@gerrit.openstack.org | 17:08 |
| jeblair | pleia2: yes | 17:08 |
| *** dina_belova has joined #openstack-infra | 17:08 | |
| pleia2 | jeblair: thanks | 17:08 |
| *** nijaba_ has quit IRC | 17:09 | |
| *** nijaba has joined #openstack-infra | 17:10 | |
| *** nijaba has quit IRC | 17:10 | |
| *** nijaba has joined #openstack-infra | 17:10 | |
| *** emagana has quit IRC | 17:15 | |
| clarkb | I noticed one issue with proposal.slave over the weekend. I think Jenkins' ssh private key was managed by hand previously so pushes to gerrit are failing. I will fix that with a puppet change | 17:15 |
| *** emagana_ has joined #openstack-infra | 17:16 | |
| jeblair | clarkb: fyi the private key showed up for the new jenkins masters (so it's not completely broken) | 17:16 |
| clarkb | jeblair: I think I am just missing the flag to the jenkins user that includes the key | 17:17 |
| *** nati_ueno has joined #openstack-infra | 17:17 | |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/config: Revert "Make gate-tempest-devstack-vm-neutron non-voting" https://review.openstack.org/40250 | 17:19 |
| *** odyssey4me has joined #openstack-infra | 17:20 | |
| dansmith | sdague: http://logs.openstack.org/37/40237/1/check/gate-grenade-devstack-vm/6809c95/logs/new/screen-s-container.txt.gz | 17:21 |
| dansmith | sdague: seems on in grenade... | 17:21 |
| jeblair | clarkb, fungi: would you mind reviewing https://review.openstack.org/#/c/39774/1 ? | 17:22 |
| clarkb | looking | 17:22 |
| *** SergeyLukjanov has joined #openstack-infra | 17:22 | |
| mordred | dstufft: convoluted questions for you | 17:22 |
| mordred | sdague: I have reproduced the situation very simply with oslo.config | 17:23 |
| mordred | dstufft: pip install oslo.config # installs 1.1.1 from pypi ; git clone git://github.com/openstack/oslo.config ; cd oslo.config ; sudo pip install -e . | 17:24 |
| mordred | dstufft: after the above | 17:24 |
| mordred | >>> oslo.config.__file__ | 17:24 |
| clarkb | jeblair: sdague does https://review.openstack.org/#/c/39774/1 mean we preinstall the requirements prior to running devstack? | 17:24 |
| mordred | '/usr/local/lib/python2.7/dist-packages/oslo/config/__init__.pyc' | 17:24 |
| *** pabelanger has quit IRC | 17:24 | |
| clarkb | or is devstack consuming the requirements list? | 17:24 |
| mordred | dstufft: so, the local thing is being installed into the easy-install.pth file | 17:24 |
| mordred | clarkb: it's slightly more complex | 17:24 |
| mordred | clarkb: in each setup_develop function call in devstack | 17:24 |
| mordred | clarkb: we call requirements:update.py | 17:25 |
| sdague | clarkb: there is a devstack change that hasn't landed yet which will do what monty says | 17:25 |
| mordred | clarkb: which will update the local requirements list in the repo we are about to install from with the appropriate lines from the global list | 17:25 |
| sdague | dansmith: so that seems like swift wasn't shut down correctly | 17:25 |
| clarkb | sdague: dansmith it is possible that the slave is being used more than once | 17:26 |
| jeblair | clarkb: strictly speaking i don't think it should matter for the actual zuul layout change; if any of what they describe is not in place, it will just be a sort of no-op run -- changes will merge if a standard devstack run succeeds. | 17:26 |
| mordred | dstufft: also, pip uninstall oslo.config is a no-op | 17:26 |
| clarkb | sdague: dansmith we fixed one problem that allowed that, but it is possible something else is also causing that to happen | 17:26 |
| *** changbl has joined #openstack-infra | 17:26 | |
| *** sarob_ has quit IRC | 17:27 | |
| clarkb | jeblair: ya, not required now. Just wondering what the gameplan was because I thought mordred had something different in mind with the pbr rawinstall stuff | 17:27 |
| openstackgerrit | Elizabeth Krumbach Joseph proposed a change to openstack-infra/config: Add httpd ssl support to git.openstack.org https://review.openstack.org/40253 | 17:27 |
| jeblair | clarkb: indeed; i like knowing gameplans too. :) | 17:27 |
| *** sarob has joined #openstack-infra | 17:27 | |
| pleia2 | hm, that didn't go well as I wanted | 17:27 |
| openstackgerrit | A change was merged to openstack-infra/config: Cleanup unneeded Jenkins vars. https://review.openstack.org/39607 | 17:27 |
| mordred | clarkb: rawinstall tests different things | 17:27 |
| clarkb | mordred: it tests ability to install, devstack with requirements will test ability to work | 17:28 |
| mordred | right | 17:28 |
| mordred | rawinstall also tests ability for us to build a mirror with a given set of requirements :) | 17:28 |
| jeblair | clarkb: according to d-g logs, the node dansmith pointed to was not used twice. | 17:28 |
| mordred | so -will adding this req break our mirror building job in a way we need to know about | 17:28 |
| openstackgerrit | David Caro proposed a change to openstack-infra/jenkins-job-builder: Added matrix trigger option to email-ext publisher https://review.openstack.org/40214 | 17:29 |
| sdague | jeblair: I've seen fails like this before, I wonder if swift in stable added a new service that doesn't get shut down | 17:29 |
| clarkb | jeblair: +2 from me. I haven't approved it to give fungi a chance to look at it | 17:29 |
| clarkb | jeblair: sdague but feel free to approve if you are in a rush | 17:29 |
| mordred | clarkb: the patch above from sdague sets us up for the other change sdague and I are working on | 17:31 |
| mordred | oh yeah - basically what jeblair said | 17:31 |
| *** ruhe has joined #openstack-infra | 17:31 | |
| *** pcrews has joined #openstack-infra | 17:32 | |
| *** sarob has quit IRC | 17:32 | |
| openstackgerrit | Elizabeth Krumbach Joseph proposed a change to openstack-infra/config: Add httpd ssl support to git.openstack.org https://review.openstack.org/40253 | 17:33 |
| *** gyee has joined #openstack-infra | 17:36 | |
| *** boris-42 has joined #openstack-infra | 17:36 | |
| harlowja_ | qq, is there any good set of slides that i can refer some of the yahoo people to that shows how awesome the CI in openstack is?? like something that explains the setup and stuff, would like to send something to them, but not sure if anything exists :) | 17:37 |
| mordred | harlowja_: we have _many_ slides | 17:37 |
| mordred | from what pov do you want? | 17:38 |
| harlowja_ | hmmm, higher level i guess to start | 17:38 |
| harlowja_ | like arcitecture maybe | 17:38 |
| mordred | harlowja_: http://docs.openstack.org/infra/publications/overview/ | 17:38 |
| harlowja_ | thx | 17:38 |
| mordred | might be a good first step | 17:38 |
| harlowja_ | def | 17:38 |
| harlowja_ | that is :) | 17:38 |
| harlowja_ | thx mordred | 17:38 |
| mordred | harlowja_: we could also probably arrange for one of us to come out and do the dog and pony show at some point | 17:39 |
| harlowja_ | hehe | 17:39 |
| harlowja_ | possibly | 17:39 |
| *** sarob has joined #openstack-infra | 17:39 | |
| harlowja_ | who's the pony this time? | 17:39 |
| mordred | whoever you want it to be ;) | 17:39 |
| *** vogxn has quit IRC | 17:39 | |
| harlowja_ | lol | 17:39 |
| *** sarob has joined #openstack-infra | 17:39 | |
| openstackgerrit | Clark Boylan proposed a change to openstack-infra/config: Add Jenkins SSH key to proposal.slave. https://review.openstack.org/40254 | 17:39 |
| clarkb | jeblair: ^ that should fix the ssh key situation on proposal.slave.o.o | 17:40 |
| harlowja_ | mordred i'll see how this goes, might take u up on that :) | 17:40 |
| mordred | sdague: pip uninstsall _before_ the oslo.config seup_develop seems to work | 17:41 |
| sdague | mordred: yeh, I've got that in the new grenade patch | 17:42 |
| sdague | hopefully with enough permissions this time | 17:42 |
| mordred | sdague: and I think I'm somewhat comfortable with the approach since what we're doing, namely, testing an upgrade from an old version installed via pip to a new version installed from trunk with setup.py develop is enough of a shift in approach that it not working naturally is not shocking | 17:43 |
| sdague | sure | 17:43 |
| sdague | it still seems like this is a pip bug though | 17:43 |
| *** burt has quit IRC | 17:43 | |
| mordred | sdague: also, once we're doing havana to icehouse, the structure of the install will be the same, and oslo.config for havana will be coming from stable.havana, so it should be less crazy | 17:43 |
| sdague | especially as you can't uninstall it | 17:43 |
| mordred | sdague: I _totally_ agree | 17:43 |
| mordred | sdague: I'd like to run through it with dstufft and see what the chances are of getting the pattern fixed in pip | 17:44 |
| sdague | I sometimes feel like openstack is a giant icebreaker going through the arctic seas smashing into an through python bugs the whole way | 17:44 |
| mordred | ya | 17:44 |
| sdague | which I guess is a good thing for everyone | 17:45 |
| sdague | just never fun to be the one who hits the bugs first :) | 17:45 |
| mordred | sdague: btw - on the 'stop installing python packages from apt' - I'm also on board with never installing from pip | 17:45 |
| mordred | it's purely the mix of packages + pip on the same machine that seems to be just not designed to work | 17:45 |
| mordred | so, I also think that an approach where we had apt and yum repos available built from requirements.txt would be, well, extra work, but also something we should be able to expect to actually operate predictably | 17:46 |
| *** Guest58376 is now known as Vivek | 17:47 | |
| mordred | harlowja_: which is anvil now? all pip? or all pkgs? | 17:48 |
| harlowja_ | all rpms | 17:48 |
| *** Vivek is now known as Guest84661 | 17:48 | |
| mordred | nod | 17:48 |
| harlowja_ | not strongly tied to rpms, but thats just what we've implemented | 17:48 |
| sdague | it's unfortunate that the python install tools slam into regular directories | 17:48 |
| harlowja_ | ya | 17:48 |
| mordred | sure - just wondering which route of pain and death you'd done down | 17:48 |
| harlowja_ | not 'deb' route yet :-p | 17:48 |
| harlowja_ | rpm route hasn't been so bad | 17:49 |
| mordred | sdague: dstufft is writing a PEP about that, btw | 17:49 |
| *** Guest84661 is now known as VivekVC | 17:49 | |
| harlowja_ | anvil did have ubuntu support, but it wasn't really being maintained so i took it out for now | 17:49 |
| mordred | sdague: to call out the need for python and distro packaging to be able to co-exist | 17:49 |
| *** VivekVC has quit IRC | 17:49 | |
| *** VivekVC has joined #openstack-infra | 17:49 | |
| *** VivekVC is now known as Vivek | 17:49 | |
| sdague | mordred: it is one of the things I like about ruby's gem, it installs into /var/lib/gems, so out of the way. It also allows for version specification on import | 17:49 |
| mordred | sdague: yeah. perl have support baked in too | 17:50 |
| mordred | the real problem comes in that python upstream doesn't have two different places for distro and non-distro add-on libraries | 17:50 |
| clarkb | cabal too | 17:50 |
| harlowja_ | just there are in openstack always going to be annoyances like libvirt bindings | 17:50 |
| mordred | yup | 17:51 |
| mordred | ah libvirt | 17:51 |
| harlowja_ | lol | 17:51 |
| mordred | so - the biggest problem we'd fact, if we went down the route of making packages from our own software, and only installing anywhere via distro packages, are things that need newer versions of libs than what's in the distro | 17:52 |
| mordred | because we have in the past maintained apt repos with backports of python dependencies | 17:52 |
| mordred | and it was a nightmare | 17:52 |
| *** apevec has joined #openstack-infra | 17:52 | |
| harlowja_ | just make them on the fly :) | 17:52 |
| mordred | but waiting on the distro means a 6 month cycle | 17:52 |
| harlowja_ | not for RHEL | 17:53 |
| mordred | harlowja_: yeah. that one is tempting | 17:53 |
| harlowja_ | thats like a 3+ year cycle ;) | 17:53 |
| mordred | harlowja_: if only upstream packaging sources were consumable in a predictable manner | 17:53 |
| *** odyssey4me has quit IRC | 17:53 | |
| mordred | harlowja_: how do you determine the appropriate distro package naming for intra-depends? | 17:53 |
| harlowja_ | so RH has a pretty basic way of appending 'python-' onto all there packages | 17:54 |
| harlowja_ | and we also have a map for weird cases | 17:54 |
| harlowja_ | *yaml map | 17:54 |
| harlowja_ | making an example, one sec | 17:54 |
| mordred | ok. that answers the question | 17:54 |
| mordred | existence of manually curated exception map | 17:55 |
| harlowja_ | ya | 17:55 |
| mordred | k. cool.. good to know | 17:55 |
| harlowja_ | aka https://github.com/stackforge/anvil/blob/master/conf/distros/rhel.yaml#L8 | 17:55 |
| harlowja_ | :-p | 17:55 |
| sdague | mordred: so https://review.openstack.org/#/c/39574/ now passes, with the explicit uninstall | 17:55 |
| mordred | I mean, we already maintain logic around special cases - such as adding pyparsing and pycffi to requireents.txt even though they're transitive | 17:56 |
| mordred | sdague: w00t! | 17:56 |
| sdague | dtroyer: can you take a look at https://review.openstack.org/#/c/39574/ | 17:56 |
| mordred | sdague: then I say we land that - it at least lets us gate moving forwar | 17:56 |
| *** dina_belova has quit IRC | 17:57 | |
| mordred | sdague: btw - once this is landed, perhaps we should make the grenade gate symmetrical? | 17:57 |
| sdague | yeh, I want dtroyer to sanity check it first | 17:57 |
| harlowja_ | then mordred we can introspect the yum repositories (very useful that yum is in python) to do matching and all that | 17:57 |
| sdague | mordred: sure, once the other devstack thing lands | 17:57 |
| mordred | sdague: oh yeah - no, I want to get solid first :) | 17:57 |
| sdague | I'd like to get this finished for reals first | 17:57 |
| mordred | sdague: o - I wrote the update projects stuff | 17:58 |
| harlowja_ | mordred apt is also in python, so u can likely make a tiny program to do the yumfind like stuff there also | 17:58 |
| *** odyssey4me has joined #openstack-infra | 17:58 | |
| harlowja_ | just not sure about that, ha | 17:58 |
| mordred | sdague: https://review.openstack.org/#/c/40068/ | 17:58 |
| *** Ryan_Lane has joined #openstack-infra | 17:58 | |
| sdague | man... we're past 40k reviews? | 17:58 |
| mordred | oh yeah | 17:58 |
| clarkb | sdague: yes, and we failed at getting review 40k | 17:58 |
| harlowja_ | lol | 17:59 |
| mordred | I tried really hard | 17:59 |
| sdague | heh | 17:59 |
| openstackgerrit | Elizabeth Krumbach Joseph proposed a change to openstack-infra/config: Add git-daemon to cgit server. https://review.openstack.org/36593 | 17:59 |
| sdague | dansmith tried hard to get 30k, and missed it as well | 17:59 |
| clarkb | hmm /me rereviewes 40068 it seems important | 17:59 |
| dansmith | I missed it by 2, IIRC | 18:00 |
| sdague | yeh, that sounds right | 18:00 |
| clarkb | mordred: does https://review.openstack.org/#/c/40068/1..2/modules/jenkins/files/slave_scripts/propose_requirements_update.sh do something that `git diff --quiet` doesn't? | 18:00 |
| harlowja_ | mordred in the anvil tools, py2rpm (which also is responsible for making pips -> rpms) has a option to just convert the damn package names, so thats how we are using it in most of the rest of anvil to do the name conversion (providing said tool the name mapping) | 18:01 |
| harlowja_ | ./tools/py2rpm --convert pip setuptools | 18:01 |
| harlowja_ | Requires: python-pip | 18:01 |
| harlowja_ | Requires: python-setuptools | 18:01 |
| harlowja_ | for exampel | 18:01 |
| openstackgerrit | Elizabeth Krumbach Joseph proposed a change to openstack-infra/config: Add git-daemon to cgit server. https://review.openstack.org/36593 | 18:01 |
| harlowja_ | anyway, back to my other work, try anvil out if u want, seems like a larger discussion around packaging/pips though :) | 18:01 |
| clarkb | harlowja_: does it name the pip executable wrong too? | 18:02 |
| mordred | clarkb: maybe not? should I just use git diff --quiet? | 18:02 |
| harlowja_ | clarkb, lol | 18:02 |
| harlowja_ | actually clarkb i think the new packages on epel are actually putting binaries for 'pip-python, pip' | 18:02 |
| harlowja_ | so they are fixing that one :-p | 18:02 |
| clarkb | harlowja_: I ask because our centos nodes symlink /usr/bin/pip to whatever centos installs it as | 18:02 |
| clarkb | nice | 18:02 |
| clarkb | mordred: I think we use git diff --quiet elsewhere | 18:02 |
| mordred | clarkb: great. I'll update it | 18:03 |
| harlowja_ | clarkb ya i never understood that, was it just to be cool, idk | 18:03 |
| clarkb | mordred: for me at least that is easier to understand that a grep and test | 18:03 |
| mordred | harlowja_: it was because there was a pre-existing perl program called pip | 18:03 |
| harlowja_ | ah | 18:03 |
| harlowja_ | well there u go, learn something new everyday | 18:03 |
| harlowja_ | i just thought it was to be diffferent :-P | 18:03 |
| mordred | clarkb: if git diff --quiet ; then | 18:04 |
| mordred | ? | 18:04 |
| *** Ryan_Lane has quit IRC | 18:04 | |
| clarkb | ya | 18:04 |
| clarkb | mordred: er | 18:04 |
| clarkb | mordred: I think you may need to invert it | 18:04 |
| clarkb | mordred: ya, exit code 1 if there are differences and you want to do stuff if there are differences | 18:05 |
| mordred | yup | 18:05 |
| mordred | $ if ! git diff --quiet ; then echo "changes" ; fi | 18:05 |
| mordred | changes | 18:05 |
| openstackgerrit | Monty Taylor proposed a change to openstack-infra/config: Update projects when requirements change https://review.openstack.org/40068 | 18:05 |
| *** ruhe has quit IRC | 18:05 | |
| jeblair | mordred: can you do a shallow clone? | 18:06 |
| mordred | jeblair: can I? I'll be submitting back up to gerrit | 18:06 |
| dtroyer | sdague: looks good. two minor comments but they're not blockers | 18:06 |
| mordred | jeblair: I'm not sure i've ever tried submitting changes to gerrit on a shallow clone - but I'm more than happy to make that change | 18:07 |
| jeblair | mordred: my understanding is that your HEAD commit should be okay; i think it's worth trying. | 18:07 |
| clarkb | will git review complain if it looks to see how far ahead of $branch you are? | 18:08 |
| jeblair | could always not use git review | 18:08 |
| mordred | jeblair: git clone --help says you can't push from a shallow clone | 18:08 |
| sdague | dtroyer: ok, your call. you want me to respin, or push it now and I'll put out a fix patch? | 18:08 |
| jeblair | mordred: ok then | 18:08 |
| mordred | jeblair: I actually toyed with the idea of trying to use devstack-gate nodes ... but then realized that those don't have the key :) | 18:10 |
| dtroyer | sdague: clean it up if you have time, I always forget to come back around and clean things up like this | 18:10 |
| *** nijaba_ has joined #openstack-infra | 18:10 | |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/config: testing https://review.openstack.org/40258 | 18:10 |
| jeblair | mordred: ^ from shallow clone. | 18:10 |
| *** nijaba has quit IRC | 18:11 | |
| mordred | jeblair: AWESOME | 18:11 |
| jeblair | mordred: also, istr we wanted to have cached copies of the repos on all jenkins nodes; but i don't think anyone has done that. | 18:11 |
| mordred | jeblair: I will make that change | 18:11 |
| sdague | dtroyer: will do | 18:11 |
| jeblair | mordred, clarkb: also, that was using git review. | 18:11 |
| clarkb | cool | 18:11 |
| mordred | jeblair: I'll put cached copies on the slaves on my tdl | 18:11 |
| mordred | jeblair: git clone --shallow=1 ? | 18:12 |
| jeblair | depth=1 | 18:12 |
| mordred | facepalm. yes. that's what I meant :) | 18:12 |
| jeblair | ok, back to my jenkinses. | 18:12 |
| openstackgerrit | Monty Taylor proposed a change to openstack-infra/config: Update projects when requirements change https://review.openstack.org/40068 | 18:12 |
| mordred | jeblair: my precious, my preciousssss | 18:12 |
| mordred | sdague: btw - I've been purposely not landing any requirements changes in order to wait for the gating | 18:13 |
| clarkb | mordred: https://review.openstack.org/#/c/40254/1 you will need that for 400068 | 18:14 |
| mordred | sdague: also, I had an idea for an additional gate check for requrements... | 18:14 |
| harlowja_ | mordred a PBR question, is it recommended now that everyone use the python setup.py test functionality that exists | 18:14 |
| *** Ryan_Lane has joined #openstack-infra | 18:14 | |
| sdague | mordred: ok, we should have it all today | 18:14 |
| mordred | sdague: if we put a hook in devstack somewhere that would allow us to instal things after upgrading setuptools but before installing things | 18:14 |
| harlowja_ | btw thx for PBR, a+ ;) | 18:15 |
| mordred | sdague: then we could write a thing that would install any _new_ requirement in the file | 18:15 |
| mordred | to make sure that adding it doesn't screw up other things | 18:15 |
| mordred | but | 18:15 |
| mordred | now that I've said that | 18:15 |
| mordred | nevermind | 18:15 |
| mordred | we'll know that for sure once someone adds it to a project | 18:15 |
| sdague | dtroyer: ok latest round of grenade patch is up | 18:16 |
| dkranz | mordred DO you think you could give a quick review for https://review.openstack.org/#/c/39291/3 (doc auto-gen in tempest) | 18:17 |
| dtroyer | sdague: +2…shall I push it on in? | 18:17 |
| *** sandywalsh has quit IRC | 18:17 | |
| dtroyer | mordred: I'm thinking I want to extract the python prereq install like I did the packages in tools/install_prereqs.sh. That way it can be used outside of stack.sh to get things right beforehand | 18:19 |
| sdague | dtroyer: yes please | 18:19 |
| sdague | plus the one below it that we need (should be in the deps list) | 18:19 |
| *** pabelanger has joined #openstack-infra | 18:19 | |
| *** dina_belova has joined #openstack-infra | 18:20 | |
| openstackgerrit | Elizabeth Krumbach Joseph proposed a change to openstack-infra/config: Add httpd ssl support to git.openstack.org https://review.openstack.org/40253 | 18:21 |
| fungi | mordred: added a couple inline comments to 40068 but lgtm | 18:22 |
| *** krtaylor has quit IRC | 18:26 | |
| openstackgerrit | A change was merged to openstack-infra/config: Add Jenkins SSH key to proposal.slave. https://review.openstack.org/40254 | 18:26 |
| clarkb | where are we on reenabling neutron tests? | 18:29 |
| openstackgerrit | Jeremy Stanley proposed a change to openstack-infra/config: add devstack gate to requirements changes https://review.openstack.org/39774 | 18:29 |
| fungi | sdague: ^ rebased for a merge conflict | 18:29 |
| fungi | clarkb: https://review.openstack.org/40250 | 18:29 |
| clarkb | sdague: nati_ueno can I approve ^ ? | 18:31 |
| *** sarob has quit IRC | 18:31 | |
| *** odyi has quit IRC | 18:31 | |
| *** sarob has joined #openstack-infra | 18:31 | |
| nati_ueno | clarkb: I think we can approve it. Failure rate still looks good https://gist.github.com/nati/6158178 | 18:31 |
| *** vipul is now known as vipul-away | 18:32 | |
| sdague | clarkb: go for it | 18:32 |
| clarkb | done | 18:33 |
| * fungi grips the safety bar tightly | 18:33 | |
| bodepd | can I delete a branch? | 18:33 |
| bodepd | in gerrit? | 18:33 |
| bodepd | I tried and it failed | 18:33 |
| fungi | bodepd: you tried pushing an empty local refname to the remote name? | 18:34 |
| clarkb | bodepd: I think you need extra permissions. force push iiirc | 18:34 |
| *** sandywalsh has joined #openstack-infra | 18:34 | |
| bodepd | I can try that | 18:34 |
| bodepd | nah, I think it's a permissions issue | 18:35 |
| *** fbo_away is now known as fbo | 18:35 | |
| bodepd | fungi: I tried that and I tried the push --delete | 18:35 |
| bodepd | both resulted in the same error (as did --force) | 18:35 |
| jeblair | bodepd: also try the web ui at https://review.openstack.org/#/admin/projects/ | 18:35 |
| *** sarob has quit IRC | 18:35 | |
| bodepd | ! [remote rejected] grizzly (can not delete references) | 18:35 |
| *** dprince has quit IRC | 18:35 | |
| openstack | bodepd: Error: "remote" is not a valid command. | 18:35 |
| openstackgerrit | A change was merged to openstack-infra/config: Revert "Make gate-tempest-devstack-vm-neutron non-voting" https://review.openstack.org/40250 | 18:36 |
| sdague | fungi: cool, I've got another one sitting on top of that | 18:36 |
| sdague | I'll rebase it as well here in a second | 18:37 |
| fungi | sdague: oops, i'd have done that one too if i'd noticed | 18:37 |
| *** odyi has joined #openstack-infra | 18:37 | |
| * fungi curses that gerrit only expands the dependencies section when there's a pending parent, but not for a child | 18:37 | |
| sdague | hmmm.... jenkins fail again - https://jenkins.openstack.org/job/gate-grenade-devstack-vm/24647/ | 18:37 |
| *** boris-42 has quit IRC | 18:38 | |
| clarkb | fungi: and automagically scrolls below the commit message | 18:39 |
| fungi | clarkb: yes, i reserve separate curses for that behavior | 18:39 |
| jeblair | sdague: no it's not | 18:39 |
| sdague | jenkins is gone very sad | 18:39 |
| sdague | jeblair: no? | 18:39 |
| jeblair | sdague: that job was aborted | 18:39 |
| sdague | ok, what aborted it? | 18:40 |
| jeblair | sdague: so when zuul knows that it's useless to continue running tests for a change, it aborts those tests | 18:40 |
| jeblair | sdague: note on http://status.openstack.org/zuul/ it's two changes behind openstack/heat38467,2 which failed python27 tests | 18:40 |
| sdague | ok, so ZUUL showed it as an ERROR | 18:40 |
| jeblair | sdague: the change ahead of it has several tests aborted | 18:40 |
| jeblair | sdague: jenkins reported back to zuul a FAILURE result. sometimes based on exactly when in the process jenkins aborts a job, it returns 'failure' instead of 'aborted'. | 18:41 |
| *** boris-42 has joined #openstack-infra | 18:41 | |
| jeblair | sdague: jenkins aborting tests is... weird. we're lucky it works at all. | 18:41 |
| clarkb | jeblair: sdague I think it has to do with who ends up catching the timeout exception | 18:41 |
| sdague | ok | 18:41 |
| clarkb | s/timeout/abort/ | 18:41 |
| sdague | yeh, a heat stable looks to have been the thing that tripped the rest of the gate | 18:42 |
| clarkb | similar mechanism in place for both things | 18:42 |
| sdague | jeblair: we aren't kicking out on a single fail? | 18:42 |
| jeblair | zuul could probably mask that and continue to report 'aborted' instead. this is a case of more accuracy being less helpful. :) | 18:42 |
| jeblair | sdague: no, only merge failures | 18:42 |
| apevec | fungi, jeblair - what made all periodic-tempest-* fail? https://jenkins.openstack.org/job/periodic-tempest-devstack-vm-stable-grizzly/126/ | 18:43 |
| jeblair | sdague: i think* it's still the case that failures ahead often cause false failures behind. | 18:43 |
| apevec | ./safe-devstack-vm-gate-wrap.sh fails w/o output? | 18:44 |
| jeblair | sdague: * (i don't have data to back that up; just noting observations) | 18:44 |
| sdague | jeblair: ok, I thought we could pull out and restart the rest of the queue on first fail | 18:44 |
| clarkb | apevec: yes the wrap script is quiet | 18:44 |
| clarkb | apevec: however, we capture what it is doing and stick it in a log file | 18:44 |
| jeblair | sdague: the head, yes, and we do that, but not anything behind the head, because things ahead may not have failed yet | 18:44 |
| clarkb | apevec: http://logs.openstack.org/periodic/periodic-tempest-devstack-vm-stable-grizzly/devstack-precise-hpcloud-az3-182671/126/logs/devstack-gate-setup-workspace-new.txt | 18:44 |
| jeblair | apevec: i just tracked that down, it's my fault | 18:44 |
| sdague | oh, ok, I thought it was resolved to change that | 18:45 |
| *** melwitt has joined #openstack-infra | 18:45 | |
| sdague | because, for instance, right now we're just blocking the gate for 20 minutes until the heat job finishes | 18:45 |
| jeblair | apevec: i need to reconfigure the jobs to accomodate a recent change i made; i'll push a change for that later today. | 18:45 |
| jeblair | sdague: we did not resolve to change that | 18:45 |
| sdague | ok | 18:45 |
| *** yolanda has joined #openstack-infra | 18:45 | |
| jeblair | sdague: any of the 3 jobs of the change at the head of the queue could fail still | 18:46 |
| sdague | they could | 18:46 |
| jeblair | sdague: which would invalidate the heat failure in question | 18:46 |
| sdague | but they wouldn't cause that one to pass | 18:46 |
| jeblair | sdague: so we would have to test that again | 18:46 |
| yolanda | hi, i'm trying to automate gerrit installation. Is there some way to setup an initial ssh key without adding that from the web interface? | 18:46 |
| sdague | in this case, they wouldn't :) it's a stable job, nothing ahead of it is | 18:46 |
| jeblair | sdague: we know that, but zuul doesn't. | 18:46 |
| apevec | jeblair, thanks! so I'm looking fwd to no email on stable-maint tomorrow | 18:46 |
| sdague | right, do you have a good query to tell us how often the recovery would have worked, and how often we're just waiting on fail? | 18:47 |
| clarkb | yolanda: an initial ssh key for a user? | 18:47 |
| sdague | because in my experience the wait is way more common | 18:47 |
| fungi | yolanda: the only mechanisms gerrit has for that are the webui and injecting directly into its backend database with an insert query, as far as i know | 18:47 |
| jeblair | sdague: i do not. | 18:47 |
| sdague | but I don't have the data | 18:47 |
| yolanda | sounds ugly | 18:47 |
| fungi | yolanda: we used to do the latter, pulling them from launchpad profiles. it was ugly, and didn't solve the problem of needing to stuff your key into a webui anyway | 18:48 |
| fungi | just happened to be a different webui | 18:48 |
| yolanda | so it's a manual step that every user should do, right? | 18:49 |
| fungi | yolanda: every user who wants to submit changes or use gerrit's ssh interface | 18:49 |
| fungi | yolanda: users can still comment on changes without doing that | 18:49 |
| yolanda | fungi, yes, trying to setup the gerrit ssh interface | 18:49 |
| clarkb | jeblair: what did the timer trigger in zuul break in the periodic jobs? | 18:51 |
| *** krtaylor has joined #openstack-infra | 18:51 | |
| jeblair | sdague: mordred was brainstorming about the possibility of having zuul start running tests assuming that (in this example) that heat change does eventually get ejected. i think that would be useful and not unfairly penalize changes that may merge, though it's not trivial. | 18:53 |
| mordred | ++ | 18:53 |
| openstackgerrit | Jeremy Stanley proposed a change to openstack-infra/config: add missing neutron-full job to neutron gate https://review.openstack.org/39775 | 18:53 |
| fungi | sdague: went ahead and rebased the other one on top of it for you | 18:54 |
| *** sarob has joined #openstack-infra | 18:55 | |
| *** burt has joined #openstack-infra | 18:55 | |
| clarkb | jeblair: that would be really neat. Do you expect to test like that beyond a fail depth of 1? | 18:55 |
| mordred | clarkb: I think essentially it winds up creating a tree of changes | 18:56 |
| clarkb | mordred: yes, which is problematic with a finite set of reources | 18:57 |
| fungi | i suppose with a cap on how many changes up the gate we'll branch to spawn parallel testing on failure, it could be tractable | 18:58 |
| bodepd | I tried all of those things and failed | 18:58 |
| * jeblair goes back to his jenkinses | 18:58 | |
| bodepd | perhaps I don't have the right permissions? | 18:58 |
| mordred | well, I think we already have built in caps on how much we can run at once | 18:59 |
| mordred | so I think we'd eventually queue things | 18:59 |
| clarkb | I hope this math is correct. If N is the number of changes in the gate then we currently run log_2 N tests compared to the branching situation described | 18:59 |
| mordred | and then a solid path of success or failure would emerge and start pruning swaths of tree | 18:59 |
| fungi | bodepd: i dug into our acls and didn't immediately see where we'd granted force push permissions to a group for any existing project, though it's possible that's the correct permission on refs/* | 18:59 |
| jeblair | we will not grant force push | 19:00 |
| fungi | bodepd: alternatively, owner permission might solve it, though we haven't done that either | 19:00 |
| fungi | bodepd: but i'll happly delete branches on request for you | 19:01 |
| jeblair | fungi: i think owner may at least make the webui work; and i'd be open to that. | 19:01 |
| *** vipul-away is now known as vipul | 19:01 | |
| fungi | right now we only have admiistrators as owner group for any project | 19:01 |
| fungi | er, administrators | 19:01 |
| fungi | bodepd: which project? we can try that out | 19:01 |
| jeblair | yeah; i'd want to understand what that would entail, but if it is just branch deletion and project description, i'd be okay with that for stackforge projects. | 19:02 |
| clarkb | ssh key is in place on proposal.slave. Once we see that has fixed the jobs that push to gerrit I will delete tx.slave | 19:02 |
| mordred | jeblair: I'd be ok with owner perms on stackforge projects | 19:03 |
| clarkb | https://review.openstack.org/Documentation/access-control.html#category_owner supports the theory | 19:03 |
| *** dprince has joined #openstack-infra | 19:05 | |
| fungi | well, it is a superset of other access though, since it allows granting/revoking any other permissions | 19:05 |
| *** sarob has quit IRC | 19:06 | |
| fungi | "Users who are members of an owner group can [...] Grant/revoke any access rights" | 19:06 |
| *** sarob has joined #openstack-infra | 19:07 | |
| *** nijaba has joined #openstack-infra | 19:11 | |
| *** nijaba has quit IRC | 19:11 | |
| *** nijaba has joined #openstack-infra | 19:11 | |
| *** sarob has quit IRC | 19:11 | |
| *** boris-42 has quit IRC | 19:11 | |
| *** Ryan_Lane has quit IRC | 19:11 | |
| jeblair | well nm then. | 19:11 |
| *** apevec has quit IRC | 19:11 | |
| *** nijaba_ has quit IRC | 19:12 | |
| clarkb | looks like you can delegate subbranch space ownership, but that doesn't really help here | 19:12 |
| *** lcestari has quit IRC | 19:13 | |
| *** lcestari has joined #openstack-infra | 19:14 | |
| *** sarob has joined #openstack-infra | 19:14 | |
| openstackgerrit | Elizabeth Krumbach Joseph proposed a change to openstack-infra/config: Add replication of git from gerrit to git.o.o https://review.openstack.org/37794 | 19:14 |
| fungi | so anyway, bodepd, the answer seems to be branch deletion will need you to ask in here, on the openstack-infra ml or via a bug filed against openstack-ci on lp | 19:14 |
| fungi | bodepd: but if you have a list of the projects+branches you want deleted, i'll be happy to take care of it | 19:15 |
| yolanda | fungi, i added my ssh keys into gerrit interface, and now i'm trying ssh -vv -p 29418 91.189.93.38 gerrit version | 19:15 |
| yolanda | but it gives me permission denied, although i pasted my correct keys | 19:15 |
| yolanda | i'm logged into gerrit using sso | 19:15 |
| fungi | yolanda: is your username in gerrit the same as your local username? | 19:15 |
| yolanda | is there something i'm missing? | 19:15 |
| fungi | yolanda: check the username you have listed at https://91.189.93.38/#/settings/ | 19:17 |
| yolanda | fungi, no, my username in gerrit is the one from sso: yolanda.robla@canonical.com | 19:17 |
| fungi | yolanda: that's the username field in gerrit, or the email address field? | 19:17 |
| fungi | yolanda: the "username" in gerrit is where you set what ssh account name you intend to use when connectnig | 19:18 |
| yolanda | oh, username is empty, i need also to add it manually? | 19:19 |
| fungi | yolanda: yes, click there and put in what ssh username you want | 19:19 |
| fungi | the way to think about it is that ssh authentication (username, keys) is bootstrapped manually from your web interface identity | 19:20 |
| openstackgerrit | Elizabeth Krumbach Joseph proposed a change to openstack-infra/config: Add replication of git from gerrit to git.o.o https://review.openstack.org/37794 | 19:21 |
| yolanda | fungi, i did a mistake with the user, but there isn't any possibility to update it? | 19:22 |
| *** Ryan_Lane has joined #openstack-infra | 19:22 | |
| fungi | yolanda: yet another shortcoming of gerrit. the only way to edit the username for an account is via an update query on the backend database | 19:23 |
| *** Ryan_Lane has quit IRC | 19:23 | |
| openstackgerrit | A change was merged to openstack-infra/config: add devstack gate to requirements changes https://review.openstack.org/39774 | 19:23 |
| fungi | yolanda: if you look in your gerrit db, the account_external_ids table should have an entry mapping your numeric account_id to an external_id like "username:something" | 19:24 |
| fungi | yolanda: you could actually just delete that row from the table and then see if the webui willlet you enter it again (rather than futzing with an update query) | 19:24 |
| openstackgerrit | A change was merged to openstack-infra/config: add missing neutron-full job to neutron gate https://review.openstack.org/39775 | 19:25 |
| fungi | sdague: ^ both merged successfully this time | 19:25 |
| sdague | nice | 19:26 |
| fungi | yolanda: apparently earlier versions of gerrit used to support changing usernames through the settings page, but then they disabled it because they implemented the ability to add user branches win which they embedded the username rather than the account id, and didn't want to have to engineer a way to automatically rename those | 19:27 |
| fungi | seems silly to me, but that's the justification i found on their discussion list a while back when i tracked it down | 19:28 |
| bodepd | fungi: I accidentally created a grizzly branch for quantum | 19:29 |
| *** Ryan_Lane has joined #openstack-infra | 19:29 | |
| fungi | bodepd: i'll kiwf | 19:29 |
| bodepd | stackforge/puppet-quantum : grizzly | 19:29 |
| yolanda | fungi, and what's the workaround? | 19:29 |
| bodepd | fungi: thanks! | 19:30 |
| *** gyee has quit IRC | 19:30 | |
| yolanda | i'm just working on a charm so i could destroy the service and redeploy it, but in the real world what should be the solution? | 19:30 |
| mordred | fungi: have you ever used the userbranch thing? | 19:30 |
| fungi | bodepd: so get rid of "grizzly" but keep "stable/grizzly" as is? | 19:30 |
| mgagne | fungi: yes | 19:31 |
| fungi | yolanda: in the real world, i connect to the mysql database with the mysql client locally from a shell account on the gerrit server, query the account_external_ids table to see what the username was for that user, then issue an sql update query to replace it with what they want | 19:32 |
| fungi | mordred: i've never tried it out, no | 19:32 |
| *** psedlak has quit IRC | 19:32 | |
| fungi | mgagne: bodepd: should be all set now | 19:32 |
| mgagne | fungi: thanks! | 19:32 |
| fungi | yolanda: it's ugly, but so far only comes up about once or twice a month | 19:33 |
| openstackgerrit | Elizabeth Krumbach Joseph proposed a change to openstack-infra/config: Add replication of git from gerrit to git.o.o https://review.openstack.org/37794 | 19:33 |
| *** mriedem has quit IRC | 19:34 | |
| *** boris-42 has joined #openstack-infra | 19:35 | |
| *** Ryan_Lane has quit IRC | 19:35 | |
| *** sandywalsh has quit IRC | 19:36 | |
| *** ArxCruz has joined #openstack-infra | 19:37 | |
| *** ruhe has joined #openstack-infra | 19:38 | |
| bodepd | fungi: thanks! | 19:39 |
| jeblair | in a brand new jenkins installation it creates a file called "secret.key.not-so-secret" | 19:41 |
| jeblair | that just about sums up the jenkins security model | 19:41 |
| *** ruhe has quit IRC | 19:41 | |
| *** dina_belova has quit IRC | 19:43 | |
| fungi | ha | 19:43 |
| fungi | that's classic | 19:43 |
| mordred | jeblair: fantastic | 19:44 |
| *** sandywalsh has joined #openstack-infra | 19:45 | |
| *** vijendar has quit IRC | 19:48 | |
| openstackgerrit | Elizabeth Krumbach Joseph proposed a change to openstack-infra/config: Add replication of git from gerrit to git.o.o https://review.openstack.org/37794 | 19:49 |
| * pleia2 mutters about aligned rockets | 19:49 | |
| *** mriedem has joined #openstack-infra | 19:49 | |
| mordred | sdague: the grenade changes landed, I'm rechecking https://review.openstack.org/#/c/35705/ | 19:49 |
| jeblair | https://jenkins01.openstack.org/ and https://jenkins02.openstack.org/ exist | 19:54 |
| jeblair | with a basic version of the global config xml files i manually copied into place (i have that in a tarball, so we can work on puppetizing it) | 19:55 |
| fungi | pleia2: see security-related comment on the https change | 19:56 |
| fungi | pleia2: other than that, lgtm | 19:56 |
| *** vipul is now known as vipul-away | 19:56 | |
| fungi | jeblair: too awesome! | 19:56 |
| *** vipul-away is now known as vipul | 19:56 | |
| pleia2 | fungi: thanks! | 19:56 |
| jeblair | working on jenkins-jobs now | 19:56 |
| openstackgerrit | Elizabeth Krumbach Joseph proposed a change to openstack-infra/config: Add httpd ssl support to git.openstack.org https://review.openstack.org/40253 | 19:57 |
| pleia2 | hm, actually | 19:58 |
| fungi | pleia2: puppet lint rejects your offering on 37794 | 19:58 |
| *** dhouck has quit IRC | 19:58 | |
| fungi | oh, puppet syntax actually | 20:00 |
| openstackgerrit | Elizabeth Krumbach Joseph proposed a change to openstack-infra/config: Add httpd ssl support to git.openstack.org https://review.openstack.org/40253 | 20:00 |
| pleia2 | there, fixed up all ssl permissions | 20:00 |
| pleia2 | fungi: hrm, reviewing | 20:00 |
| fungi | pleia2: you have an extra <% end -%> in replication.config.erb | 20:01 |
| fungi | seems to be the gist of its complaint | 20:01 |
| pleia2 | ah, good eyes | 20:01 |
| fungi | i only know to recognize that because i've done it myself on at least a few occasions | 20:02 |
| openstackgerrit | Elizabeth Krumbach Joseph proposed a change to openstack-infra/config: Add replication of git from gerrit to git.o.o https://review.openstack.org/37794 | 20:03 |
| pleia2 | too much copying around of things in that file :) | 20:03 |
| pleia2 | should be better now | 20:03 |
| fungi | copy+paste... both the cause of and solution to most coding errors | 20:04 |
| *** SergeyLukjanov has quit IRC | 20:07 | |
| *** nijaba has quit IRC | 20:09 | |
| fungi | pleia2: cgit@git.openstack.org:/var/lib/git/${name}.git works with no protocol specifier (e.g., ssh://)? | 20:11 |
| *** nijaba has joined #openstack-infra | 20:11 | |
| pleia2 | fungi: good question | 20:11 |
| *** vipul is now known as vipul-away | 20:11 | |
| fungi | wasn't sure if that had been tested | 20:11 |
| pleia2 | fungi: I tested one version of it that I had, but it's been several revisions+weeks so I better give it a go again | 20:12 |
| fungi | pleia2: looks like it should be fine the way you have it... https://review.openstack.org/Documentation/config-replication.html | 20:13 |
| pleia2 | fungi: that was it - github is ssh right? | 20:13 |
| pleia2 | I think I just copied what it was doing there | 20:14 |
| *** boris-42 has quit IRC | 20:14 | |
| fungi | yeah, seems that way | 20:14 |
| pleia2 | yeah, remember reading the docs now :) | 20:14 |
| fungi | lgtm! | 20:14 |
| pleia2 | and I'm assuming that ${name} will expand properly | 20:15 |
| pleia2 | I don't have a setup quite this complex in testing | 20:15 |
| *** vipul-away is now known as vipul | 20:17 | |
| yolanda | fungi, i was able to log in and use ssh commands, but i need to have the privilege to create-account. How can i set it up, still more user interface needed? | 20:18 |
| fungi | yolanda: https://your.gerrit.server/#/admin/projects/All-Projects,access | 20:20 |
| *** vijendar has joined #openstack-infra | 20:20 | |
| yolanda | so manual setup is needed | 20:21 |
| fungi | yolanda: in that acl we have a "Global Capabilities" section at the top with an "Administrate Server" permission delegated to a group called "Administrators" and the members of that group are able to create-account | 20:21 |
| fungi | yolanda: if you look on the filesystem, parallel to the other org/project directories you'll see a project called "All-Projects" | 20:22 |
| yolanda | fungi, i cannot see that screen, i only see the reference: refs/*, but no global capabilities section | 20:22 |
| fungi | yolanda: okay, it's likely a chicken-and-egg issue there... you probably have an initial gerrit administrator account of some kind which can modify that | 20:23 |
| fungi | yolanda: you can probably just add your account_id as a member of Administrators with an update to the account_group_members table | 20:24 |
| yolanda | touching directly mysql? | 20:27 |
| fungi | on our gerrit, Administrators is group_id 1 (that's probably typical of most any gerrit server), but you ca confirm on yours by looking in the account_group_names table | 20:27 |
| jeblair | woo! JJB is running on jenkins01 and jenkins02 | 20:28 |
| fungi | yolanda: well, normally when you set up a gerrit from scratch i think you get an administrator account initially which is expected to be used to do most of the initial configuration in the webui. it's been a little while since i built one from google's instructions, though zaro may be able to confirm | 20:28 |
| mordred | jeblair: woot | 20:28 |
| fungi | jeblair: sounds very close | 20:28 |
| mordred | fungi, yolanda whoever is the first person to log in to the gerrit ttw gets admin access mgically | 20:28 |
| clarkb | jeblair: exciting | 20:29 |
| yolanda | mordred, i use sso, and i've logged for first time with my user, but no admin rights | 20:29 |
| fungi | oh, right, mordred just built one recently too | 20:29 |
| yolanda | i'm on anonymous and registered users group only | 20:30 |
| jeblair | clarkb, fungi, mordred: i'm going to be slowly taking persistent slaves offline one at a time to move them to the new masters. | 20:30 |
| clarkb | jeblair: ok | 20:30 |
| mordred | jeblair: I suppoert you in that | 20:30 |
| clarkb | jeblair: any particular partitioning scheme you want to use for that? | 20:30 |
| jeblair | (i'll leave a reason note in the webui as i do that) | 20:30 |
| fungi | jeblair: sounds like delicate surgery--let me know if you need any assistance on parts of that | 20:30 |
| jeblair | clarkb: how about odds to jenkins01 and evens to jenkins02? | 20:31 |
| clarkb | jeblair: wgm | 20:31 |
| fungi | that makes sense to me (and then divvy up special-purpose slaves at random) | 20:31 |
| clarkb | *wfm | 20:31 |
| jeblair | that'll let us use our current crop on the two masters and leave room for expansion | 20:31 |
| jeblair | fungi: i was planning on leaving special purpose slaves attached to jenkins.o.o for now | 20:32 |
| fungi | that works too | 20:32 |
| jeblair | fungi: and in the long run, continue to segregate them for improved security | 20:32 |
| clarkb | ++ | 20:32 |
| jeblair | (but possibly move them to a replacement jenkins.o.o) | 20:32 |
| fungi | definitely | 20:32 |
| jeblair | anyway, if we expand to 3 test-running masters, we'll need a new system, but this'll hold us for a bit i hope. | 20:32 |
| fungi | heralding back to our "slaves are trusted by jenkins to run arbitrary code on their master" discussion | 20:32 |
| fungi | did anybody on #jenkins follow up to your questions about that? | 20:33 |
| fungi | i never saw a response if so | 20:33 |
| jeblair | fungi: i didn't see one. | 20:33 |
| fungi | :( | 20:33 |
| fungi | well, we have a near-term workaround of putting untrusted slaves on untrusted masters, and a longer-term possibility of moving the sensitive slaves off jenkins entirely | 20:34 |
| *** yolanda has quit IRC | 20:34 | |
| fungi | so i am less worried now | 20:34 |
| jeblair | yep | 20:35 |
| clarkb | is there any concern that a master could become a gearman cleint? | 20:36 |
| clarkb | in addition to having just a bunch of workers? | 20:36 |
| clarkb | overall I think it is a better security situation but something like jenkins becoming a gearman client would worry me | 20:37 |
| jeblair | clarkb: it already is | 20:37 |
| clarkb | oh | 20:37 |
| jeblair | clarkb: that is a concern, that a master could take jobs it shouldn't be taking... | 20:37 |
| jeblair | clarkb: however, if it doesn't have the creds to do something like publish an artifact | 20:38 |
| jeblair | clarkb: the security situation isn't that bad (but annoying, because our job didn't get run) | 20:38 |
| jeblair | clarkb: we could address that (later) by segregating the gearman queues | 20:38 |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/config: Add new jenkins masters to zuul firewall https://review.openstack.org/40280 | 20:40 |
| jeblair | clarkb, fungi, mordred: speaking of which, could i get a speedy review on that brown bag? ^ | 20:40 |
| clarkb | yup | 20:40 |
| * mordred look | 20:40 | |
| jeblair | thanks | 20:40 |
| clarkb | I am going to kick into review mode now that lunch has been had. I will start with that change | 20:40 |
| clarkb | we really need a better story for firewall management across our "clusters" of things | 20:41 |
| clarkb | but that lgtm | 20:41 |
| clarkb | maybe that would be a good first candidate for salting? | 20:41 |
| fungi | oh, mordred was looking too. sorry, i already approved | 20:41 |
| mordred | fungi: all good | 20:42 |
| openstackgerrit | A change was merged to openstack-infra/config: Add new jenkins masters to zuul firewall https://review.openstack.org/40280 | 20:43 |
| clarkb | notmyname: had a chance to think about https://review.openstack.org/#/c/39388/ ? | 20:43 |
| *** dina_belova has joined #openstack-infra | 20:44 | |
| notmyname | clarkb: I actually don't remember the last answer I gave you ;-) | 20:44 |
| clarkb | notmyname: you said you would catch up on work and hopefully let me know by the end of last week :) | 20:44 |
| notmyname | clarkb: ah, ok :-) | 20:44 |
| clarkb | fungi: mordred https://review.openstack.org/#/c/39513/ want to look at that and make sure I didn't butcher the logo too horribly? | 20:45 |
| *** fbo is now known as fbo_away | 20:46 | |
| *** odyssey4me2 has joined #openstack-infra | 20:46 | |
| clarkb | mordred: for https://review.openstack.org/#/c/33204/ how comfortable are we with making that change? eg will we have to do a bunch of manual surgery to fix setuptools before devstack-gate will update itself? | 20:46 |
| russellb | heads up guys ... i'm about to push a novaclient release. we haven't done one in months. ping me if something crops up that could be caused by it so i can help address it asap. | 20:46 |
| mordred | russellb: k | 20:46 |
| mordred | russellb: I _think_ we should be ok | 20:47 |
| mordred | russellb: oh - hey, you know what? | 20:47 |
| russellb | yeah i think so too, but figured it didn't hurt to mention it | 20:47 |
| *** fbo_away is now known as fbo | 20:47 | |
| mordred | russellb: if you're going to release on, removing d2to1 from the requirements and the setup_requires would be nice | 20:47 |
| mordred | I was just about to start pushing out patches for that | 20:47 |
| russellb | mordred: ok, i can wait | 20:47 |
| fungi | we always love heads up on potential breakage, no matter how unlikely | 20:47 |
| mordred | lemme send you a patch real quick | 20:47 |
| russellb | k | 20:47 |
| russellb | fungi: :-) | 20:47 |
| notmyname | clarkb: I'm looking back over the recent patches that have merged. I don't see the test running | 20:48 |
| *** dina_belova has quit IRC | 20:48 | |
| notmyname | clarkb: eg https://review.openstack.org/#/c/39361/ | 20:49 |
| *** odyssey4me has quit IRC | 20:49 | |
| clarkb | notmyname: it isn't running in the gate, only as part of the check tests that run when you submit patches | 20:50 |
| notmyname | clarkb: ok | 20:50 |
| clarkb | notmyname: on that example change the test ran and was reported in the last comment from July 30 | 20:50 |
| *** odyssey4me2 has quit IRC | 20:50 | |
| notmyname | ah ok | 20:51 |
| fungi | reasoning being that devs will almost certainly never look at non-voting test results after approval, so they're a waste of resources to run in the gate pipeline | 20:51 |
| fungi | unless you're trying to collect post-facto statistics on them | 20:52 |
| mordred | russellb: https://review.openstack.org/40282 there's a full sync (what we're about to start running in the gate) | 20:52 |
| notmyname | clarkb: and https://jenkins.openstack.org/job/gate-swift-devstack-vm-functional/ is the aggregate page, right? | 20:53 |
| russellb | mordred: ok ... put my +2, but will wait to see tests pass, and then bug someone else to approve | 20:53 |
| clarkb | notmyname: yes, however we delete logs older than a day there | 20:54 |
| mordred | russellb: ++ thanks | 20:54 |
| clarkb | notmyname: so you are looking at an aggregate of the last 24 hours or so | 20:54 |
| *** dina_belova has joined #openstack-infra | 20:54 | |
| mordred | fungi: got a sec for an easy change that's blocking lifeless ? https://review.openstack.org/#/c/39858/ | 20:54 |
| notmyname | clarkb: ok, that's ok (makes sense for now) | 20:54 |
| jeblair | notmyname: we can make a graph of success/failure over a longer period (but can't associate those points with individual changes) | 20:54 |
| lifeless | mordred: yes that is the one | 20:54 |
| notmyname | clarkb: jeblair: start gating. +1 | 20:54 |
| fungi | clarkb: logo looks great | 20:55 |
| fungi | mordred: having a look | 20:55 |
| clarkb | notmyname: awesome. I can push that change through | 20:55 |
| notmyname | clarkb: jeblair: now, let's figure out how to make probe tests run on a patch ;-) | 20:56 |
| clarkb | mordred: https://review.openstack.org/#/c/33204/ has new comments | 20:56 |
| notmyname | and by "let's" I mean, "you guys have fun" | 20:56 |
| clarkb | unfortunately a lot has changed since you first proposed the pbr for devstack-gate change | 20:56 |
| jeblair | notmyname: refresh my memory about probe tests please? | 20:56 |
| notmyname | jeblair: between unit tests and functests. the probe tests start and stop various services in the cluster as a whole and ensure that stuff like handoffs and replication and etc work | 20:57 |
| clarkb | notmyname: change is approved. It will take a few minutes to percolate into zuul, but you will be gating on that shortly | 20:57 |
| sdague | mordred: 16 mins until the global-requirements devstack results come back | 20:58 |
| sdague | had to rebase it again | 20:58 |
| jeblair | (jenkins is nearly idle, which is frustrating my attempts to actually see if the new masters work!) | 20:58 |
| clarkb | fungi: https://review.openstack.org/#/c/35104/ is that something I should be focusing on yet? | 20:58 |
| mordred | jeblair: I can push patches if that would be helpful... | 20:58 |
| *** dina_belova has quit IRC | 20:58 | |
| openstackgerrit | A change was merged to openstack-infra/config: Vote and gate on swift functional tests. https://review.openstack.org/39388 | 20:58 |
| clarkb | mordred: you can address comments in 33204 :) | 20:58 |
| jeblair | mordred: it would, actually, especially ones with python26 tests | 20:58 |
| mordred | jeblair: I can _VERY_ easily trigger requirement update patches... | 20:58 |
| mordred | clarkb: on it | 20:58 |
| *** psedlak has joined #openstack-infra | 20:58 | |
| clarkb | jeblair: for the other launch slaves how are we setting the maximum slave numbers | 20:59 |
| *** psedlak has quit IRC | 20:59 | |
| clarkb | jeblair: did you take 96 and divide by 3? | 20:59 |
| *** ArxCruz has quit IRC | 21:00 | |
| jeblair | clarkb: i have not done that yet; but i intend to start them small to make sure they work, then up to 96/3, then 96/2. | 21:00 |
| fungi | clarkb: yeah 35104 is cool. i wanted to try spinning those tests up on some out-of-service jenkins slaves just to confirm that the nondeterministic shutil problems were mine and not intrinsic to how the tests are written, but haven't gotten to it yet (and i should feel bad) | 21:01 |
| jeblair | clarkb, fungi, mordred: the new version of jenkins has that global 'credential storage' thing; which may necessitate a change to how devstack-gate launches slaves. i'll be finding that soon. | 21:01 |
| clarkb | fungi: ok, we can land the tests and make them non voting if you are worried | 21:02 |
| fungi | clarkb: alternative tack there, i think, is that i should just propose a change to enable python26,27,33 tests on git-review first (with a no-op in place) and then recheck that change | 21:02 |
| fungi | clarkb: they're not actually being run yet anyway | 21:02 |
| mordred | jeblair: patches incoming | 21:02 |
| clarkb | fungi: I like the idea of enabling them that way and rechecking the change to add the tests | 21:02 |
| clarkb | fungi: it will make reviewing the tests easier too | 21:02 |
| mordred | jog0: ping | 21:03 |
| fungi | clarkb: also for the pbr-ization change that depends on, i think i'd like to tag a git-review release on the change ahead of it (top of master now or any trivial fixes we squeeze ahead of it at the last minute) | 21:03 |
| *** gyee has joined #openstack-infra | 21:04 | |
| jeblair | mordred: thank you https://jenkins01.openstack.org/job/gate-python-glanceclient-python26/1/ | 21:04 |
| clarkb | fungi: ++ | 21:04 |
| mordred | jeblair: w00t | 21:04 |
| mordred | jeblair: for once an automated patch bomb from me is useful! | 21:04 |
| fungi | clarkb: thinking being that the release which switches to pbr is a fairly major packaging change for the tool, so having alast known good version to point people at with no outstanding fixes pending would be good | 21:05 |
| jeblair | mordred: that job is history! it just might be the first job run as part of a distributed multi-master jenkins system. :) | 21:05 |
| mordred | maybe I'll tweet that | 21:05 |
| clarkb | fungi: indeed. The switch to pbr hasn't been uneventful, I think there is value in releasing first to have as much time to test the pbr change as possible | 21:06 |
| jeblair | and it succeeded. | 21:06 |
| fungi | clarkb: just in case it tickles problems on $random_platform like freebsd or mac osx or arch linux or who knows where some people run this | 21:06 |
| clarkb | jeblair: congrats | 21:06 |
| clarkb | I remember the scheming about this from almost a year ago | 21:06 |
| jeblair | clarkb: and now we have our preciouses | 21:06 |
| fungi | i need to run off and dine, but will beat on those things when i get back in a bit | 21:06 |
| clarkb | fungi: ok, I will hold off reviewing those two git review changes until after you have jenkins reporting test results back | 21:08 |
| zaro | jeblair: awsome! | 21:08 |
| jeblair | zaro: thanks to your plugin. :) | 21:08 |
| jeblair | mordred: want to see something cool that i did not anticipate? | 21:09 |
| mordred | jeblair: yes! | 21:09 |
| jeblair | mordred: despite the fact that these didn't work (the jobs need a small change); check out https://jenkins.openstack.org/job/periodic-tempest-devstack-vm-stable-grizzly/126/ | 21:09 |
| jeblair | mordred: it shows all the periodic tempest jobs together... | 21:09 |
| jeblair | mordred: because of the way i grouped them in the zuul layout -- they form a zuul build set. :) | 21:10 |
| clarkb | nice | 21:10 |
| mordred | neat! | 21:10 |
| clarkb | jeblair: because they all belong to tempest's periodic pipeline? | 21:10 |
| jeblair | clarkb: oh, i forgot to answer -- it looks like the way gearman plugin starts the job, the default parameter doesn't get set; so we just need to set ZUUL_BRANCH in the job itself rather than as a paremeter. | 21:11 |
| jeblair | (i like that better anyway) | 21:11 |
| jeblair | (or alternately, to have ZUUL set the branch parameter; but that needs a little more sophistication in the timer trigger) | 21:11 |
| *** nijaba_ has joined #openstack-infra | 21:11 | |
| *** thomasem has quit IRC | 21:11 | |
| jeblair | clarkb: yes (tempest periodic pipeline) | 21:11 |
| mordred | jeblair: so, I have another patch bomb loaded if you want another | 21:12 |
| *** pabelanger has quit IRC | 21:12 | |
| jeblair | mordred: not atm; i'm probably a while from the next time i need major testing; bombing isn't useful for me, but you may not want to wait. | 21:12 |
| *** nijaba has quit IRC | 21:12 | |
| clarkb | jeblair: I prefer setting it in the job too | 21:13 |
| clarkb | jeblair: I like keeping tools simple and the timer trigger shouldn't do much more than trigger on a timer imo | 21:13 |
| mordred | jeblair: ok. I'll do this one (it's updating the contents of the last one, which are slightly off) | 21:13 |
| jog0 | mordred: pong | 21:19 |
| mordred | uhm. python-swiftclient does not participate in the devstack gate | 21:21 |
| notmyname | mordred: is there a reason or is that an oversight? | 21:22 |
| clarkb | mordred: correct, I remember asking about that once and I believe the reason is it doesn't intersect anything in the gate | 21:22 |
| mordred | we install it | 21:22 |
| sdague | mordred: and we have a winner | 21:22 |
| sdague | https://review.openstack.org/#/c/35705/ | 21:22 |
| mordred | and with the new coordinated requirements gating, I think we should add it, else we're going to get burned | 21:22 |
| clarkb | mordred: I agree | 21:22 |
| *** vipul is now known as vipul-away | 21:23 | |
| mordred | sdague: AWESOME | 21:23 |
| clarkb | mordred: as that adds an intersection | 21:23 |
| sdague | dtroyer: https://review.openstack.org/#/c/35705/ you want to give it a look, and +A if you like it | 21:23 |
| mordred | jeblair: ^^ | 21:23 |
| *** woodspa has quit IRC | 21:24 | |
| openstackgerrit | Russell Bryant proposed a change to openstack-infra/reviewstats: Update nova-core membership https://review.openstack.org/40297 | 21:24 |
| mordred | jog0: so - with the new things we're doing with requirements, I think hacking 0.7 in the global reqs files is a potential issue | 21:24 |
| *** pabelanger has joined #openstack-infra | 21:24 | |
| dtroyer | sdague: I was looking at it on fedora/centos. no joy yet but I think that's still due to the pip installation problems | 21:24 |
| mordred | jog0: and we need to come up with a strategy | 21:24 |
| sdague | dtroyer: that's my guess | 21:25 |
| clarkb | mordred: earlier I suggested treating it like pep8, | 21:25 |
| clarkb | mordred: allow new rules in at the beginning of a cycle | 21:25 |
| sdague | dtroyer: I think we have to unwind fedora after this | 21:25 |
| *** dkranz has quit IRC | 21:25 | |
| mordred | jog0: because we're about to start proposing requirements update patches automatically to all of the projects every time reuqirements changes | 21:25 |
| mordred | jog0: and right now, that's going to fail for most projects because of hacking 0.7 | 21:26 |
| dtroyer | sdague: yeah, too many things at once isn't going well ;) | 21:26 |
| mordred | sdague: I agree | 21:26 |
| sdague | also, I think we need a volunteer, possibly from red hat, to help get fedora up as a devstack target | 21:26 |
| clarkb | mordred: https://review.openstack.org/#/c/35705/16/functions where does REQUIREMENTS_DIR come from? | 21:26 |
| dstufft | Fedora, slightly more broken than ubuntu with regards to pip | 21:26 |
| clarkb | mordred: that is set by a previous change to add openstack/requirements to devstack? | 21:26 |
| sdague | clarkb: lib/infra | 21:26 |
| mordred | sdague: I'll write a mailing list message about pip/pkgs | 21:27 |
| dtroyer | Ian has been doing a lot along those lines, specifically for rhel6 but I think if rhel6 is happy fedora is ahppy in general | 21:27 |
| openstackgerrit | A change was merged to openstack-infra/reviewstats: Update nova-core membership https://review.openstack.org/40297 | 21:27 |
| mordred | dtroyer: yeah - the main problem is the rpm vs. pip interaction on fedora/rhel | 21:27 |
| sdague | dtroyer: yeh, but I'd really like to know on each devstack change if it's actually going to work on fedora | 21:27 |
| sdague | so if the pg target was on fedora, that would be really handy | 21:27 |
| sdague | we do kind of officially support both latest ubuntu and fedora, but we only run integration on ubuntu | 21:28 |
| dtroyer | agreed…we've set fedora as the target and that's where I want to focus. | 21:29 |
| jeblair | perhaps we should run it on centos instead (due to the same release cycle concerns that led us to chose centos for the unit tests?) | 21:29 |
| mordred | ++ | 21:30 |
| clarkb | ++ | 21:30 |
| lifeless | stevebaker: probably worth checking the design here; then doing it - avoiding round trips | 21:30 |
| mordred | and we've already got partial centos support in the pool | 21:30 |
| clarkb | mordred: centos should work from a puppet perspective because unittest slave is a superset of the d-g slave image | 21:30 |
| mordred | stevebaker: I believe you want to add pypi-jobs to python-jobs | 21:30 |
| mordred | as in- I think you want both | 21:30 |
| mordred | clarkb: ^^ if they want to do pypi publication, that's right, right? | 21:31 |
| clarkb | we would need multiple image support in d-g. I think that exists simply by adding the ready slaves number >0 to an image in the DB | 21:31 |
| clarkb | mordred: yes | 21:31 |
| mordred | yes. it does | 21:31 |
| jeblair | clarkb: d-g has (and has used) multiple image support | 21:31 |
| mordred | we had oneiric slaves already | 21:31 |
| stevebaker | oh, ok. layout is now looking like this http://paste.openstack.org/show/43238/ | 21:31 |
| clarkb | oh ya, and it is as simple as adding the row in the DB right? | 21:31 |
| jeblair | clarkb: yep | 21:32 |
| jog0 | mordred: yeah that will make things a little tricky | 21:32 |
| sdague | jeblair: then centos would need to become a first class citizen on devstack | 21:32 |
| jeblair | clarkb: will be added with 'devstack-{imagename}' label | 21:32 |
| clarkb | stevebaker: I don't think you need the post mirror jobs | 21:32 |
| jog0 | mordred: and you want to propose the most up to date deps every time right? | 21:32 |
| stevebaker | clarkb: ok | 21:32 |
| sdague | jeblair: f19 would be much better | 21:32 |
| mordred | jog0: we do | 21:32 |
| clarkb | stevebaker: so remove lines 13 and 14 and the last character of line 12 | 21:32 |
| jeblair | sdague: how close are centos/fedora for our needs? | 21:32 |
| sdague | jeblair: I think f19 is the right target, centos is so old | 21:33 |
| sdague | there are so many work arounds you need | 21:33 |
| jog0 | mordred: and i don't like the idea of blocking until the new rules are fixed | 21:33 |
| jeblair | sdague: the tc decided to support latest ubuntu/fedora without breaking LTS/RHEL. | 21:33 |
| mordred | so... the thing with latest fedora and why it might be a problem | 21:33 |
| mordred | is the reason we ditched oneiric | 21:33 |
| clarkb | sdague: the problem with that is the distro upstreams iterate too quickly and don't support the older things long enough | 21:33 |
| jog0 | mordred: don't you need something for the general case of non-backwards compat changes? | 21:33 |
| clarkb | sdague: so you run into a situation where you have unsupported releases doing all your testing while you wait for HP/rackspace to update their image list | 21:34 |
| jeblair | sdague: in my view, that makes LTS/RHEL a reasonable LCD for tests. | 21:34 |
| mordred | that being - we like to continue testing stable releases on the thing we tested them on at release time | 21:34 |
| stevebaker | clarkb: how about this? http://paste.openstack.org/show/43239/ | 21:34 |
| clarkb | the other problem is when you are trying to test stable/folsom and it doesn't run on f20 | 21:34 |
| mordred | and f19 doesn't have the same lifespan as our stable releases | 21:34 |
| *** dprince has quit IRC | 21:34 | |
| mordred | jog0: we're about to start syncing requirements automatically in the devstack gate | 21:34 |
| clarkb | stevebaker: that looks good. Don't forget to update projects.yaml in modules/openstack_project/files/jenkins_job_builder/config | 21:35 |
| mordred | jog0: so people wanting to consume new upstream versions of things that are incompat are going to have to do work anyway | 21:35 |
| jog0 | but we don't run hacking in devstack | 21:35 |
| mordred | jog0: that's right - but hacking seems to be the only thing where the model is breaking | 21:35 |
| clarkb | stevebaker: zuul layout.yaml tells zuul which tests to run and projects.yaml tells jenkins job builder which job templates to instantiate and apply to jenkins itself | 21:35 |
| stevebaker | ok | 21:35 |
| jog0 | mordred: hrmm we could add some special log in for this case | 21:36 |
| jog0 | logic* | 21:36 |
| mordred | jog0: https://review.openstack.org/#/q/status:open+branch:master+topic:openstack/requirements,n,z fwiw | 21:37 |
| jog0 | if this is only for devstack and not unit tests then is this still an issue? | 21:37 |
| mordred | it's not a gate breaking issue | 21:37 |
| mordred | it's that we willbe proposing patches to update things | 21:37 |
| mordred | and theyu'll be breaking on pep8 | 21:37 |
| sdague | jeblair: so the problem is lots of things don't work on RHEL6. And lots of them are only fixed by RDO packages later (including replacement kernels) | 21:38 |
| sdague | I think you'll massively take a hit in dev velocity if people need to wait for RH6 RDO to fix something | 21:38 |
| mordred | maybe the answer there is that someone just has to fix the proposed changes | 21:38 |
| lifeless | mordred: btw per request I'm reviewing https://review.openstack.org/#/q/status:open+project:openstack-dev/pbr,n,z daily | 21:38 |
| mordred | lifeless: thanks. I appreciate that | 21:38 |
| jog0 | mordred: actaully yeah that would work | 21:38 |
| *** krtaylor has quit IRC | 21:38 | |
| lifeless | mordred: but everything there either failed V or has -1's already. I'm not dogpiling :). | 21:38 |
| mordred | jog0: that's what I'm doing right now on the client libs | 21:39 |
| jog0 | hacking x goes out, and auto dep patch goes out | 21:39 |
| mordred | lifeless: :) I'm deferring those for a bit to work on other things | 21:39 |
| jog0 | it braeks someone has to fix it before it merges | 21:39 |
| sdague | RHEL6 is almost as old as lucid | 21:39 |
| clarkb | jog0: mordred: I think that will be particularly painful with nova and other high volume projects | 21:40 |
| mordred | jog0: right. although also I think in general if we add new hacking checks midcycle too many times someone will kill us | 21:40 |
| clarkb | jog0: mordred: the gate will break every time hacking updates and there will be a huge rush to push fixes in and you end up in a rebase mess | 21:40 |
| mordred | o, back to what clarkb said - we shoudl probably aply the pep8 policy here | 21:40 |
| mordred | which is only rev the release once per cycle | 21:40 |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/config: Remove ZUUL_BRANCH peridic job parameter https://review.openstack.org/40301 | 21:41 |
| clarkb | sdague: I don't think we disagree that it is suboptimal. But you open yourself to a bunch of problems trying to track latest version of distros when you don't have galnce service | 21:41 |
| jog0 | clarkb mordred: yeah but that is a managable issue. or someone disables the new tests in a seperate patch | 21:41 |
| *** UtahDave has joined #openstack-infra | 21:41 | |
| sdague | mordred: so honestly, I'd wait on force updating requirements in the projects. We've solved it in the devstack side of the gate now, and I think we should discuss with a wider ptl audience to figure out the unit test side | 21:41 |
| jeblair | dtroyer: what is http://partnerweb.vmware.com/programs/vmdkimage/debian-2.6.32-i686.vmdk about? | 21:42 |
| clarkb | sdague: ubuntu and fedora both have short periods of support now. Short enough that it is possible to end up in a spot with no security updates coming from upstream | 21:42 |
| jeblair | dtroyer: i just started a wget on it and it says it will take 48 minutes to download | 21:42 |
| jog0 | clarkb: when a new hacking is cut we just propose some disable new tests that are non-trivial to fix right away patches | 21:42 |
| openstackgerrit | Steve Baker proposed a change to openstack-infra/config: Do not run python jobs for tripleo-image-elements https://review.openstack.org/39858 | 21:42 |
| jog0 | and once those merge merge new reqs patch | 21:42 |
| dtroyer | jeblair: IIRC that's the default image that should only be downloaded for vsphere (checking) | 21:43 |
| sdague | clarkb: so there is no way for us to push our own ami in? | 21:43 |
| clarkb | jog0: how is that different than what I propose? | 21:43 |
| clarkb | jog0: my solution does it with a single change to a single repo though | 21:43 |
| sdague | clarkb: fedora looks like it's giving folks a year | 21:43 |
| *** mriedem has quit IRC | 21:43 | |
| jeblair | dtroyer: yeah, that looks right | 21:43 |
| clarkb | sdague: currently our cloud providers do not give us push access to glance | 21:43 |
| jeblair | dtroyer: devstack-gate has a naive way of caching images and gets them all. | 21:44 |
| sdague | jeblair: yes, that's their "cirros like" image | 21:44 |
| jog0 | clarkb: well in devstack hacking doesn't matter, but for the actual 'pep8' tests I assume those come from local reqs | 21:44 |
| clarkb | sdague: you can do something nasty like takeover node, but that is less than ideal | 21:44 |
| sdague | clarkb: hmph, that's the suck | 21:44 |
| jeblair | it takes d-g 14 minutes to download it inside of hpcloud | 21:44 |
| stevebaker | clarkb, jeblair, getting closer? https://review.openstack.org/#/c/39858/ | 21:44 |
| sdague | that actually seems kind of crazy, when they fixing that? | 21:44 |
| mordred | sdague: they are both working on it | 21:44 |
| jog0 | mordred clarkb: I am not fully on board with hacking release only at beginning of cycle, so far we have done between milestones | 21:45 |
| sdague | mordred: so after they fixed that, would we be able to think about fedora? | 21:45 |
| *** fbo is now known as fbo_away | 21:45 | |
| jog0 | 2x a year is too infrequent for some things | 21:45 |
| clarkb | stevebaker: yup. are the python jobs not actually used in that project? If so maybe remove that line from projects.yaml? | 21:45 |
| jeblair | i'm less concerned about the timing of new releases than clarkb is | 21:45 |
| jeblair | i feel like that's a minor inconvenience to deal with after we agree on what we ought to be testing based on other reasons (such as release cycle lifetime) | 21:46 |
| clarkb | jog0: I am not advocating you release twice a year. I am advocating we pin the projects twice a year | 21:46 |
| *** rnirmal has quit IRC | 21:46 | |
| jog0 | clarkb: that is effectively the same thing | 21:46 |
| openstackgerrit | Steve Baker proposed a change to openstack-infra/config: Do not run python jobs for tripleo-image-elements https://review.openstack.org/39858 | 21:46 |
| stevebaker | clarkb: done | 21:46 |
| jog0 | clarkb: if no one uses it who cares about the releases | 21:46 |
| clarkb | jeblair: I think release cycle lifetime and length of support are related problems | 21:47 |
| *** changbl has quit IRC | 21:48 | |
| notmyname | clarkb: FWIW, running swift funcational tests locally takes around 70 seconds (on my VM). seems that jenkins takes about 10 minutes | 21:48 |
| clarkb | if length of distro support is <= length of openstack support you will have trouble even without the ability to upload your own images (maybe that is what you mean by it being more important) | 21:48 |
| clarkb | notmyname: I believe those tests are setting up devstack | 21:48 |
| clarkb | notmyname: on a single use fresh VM each time hence the additional time | 21:49 |
| *** sarob_ has joined #openstack-infra | 21:49 | |
| jog0 | clarkb: I am saying twice a year is not enogh when these are tests written just for us, but roughly 4x a year at most | 21:49 |
| notmyname | clarkb: ya, ok. (perhaps a perfect use case for openstack + docker) | 21:49 |
| jeblair | clarkb: yes; i'm a bit more concerned about the backend; there are ways we can speed up the frontend (takeovernode, apt-get dist-upgrade, glance) but it's not worth engineering if we don't _want_ to support them because they aren't compatible with our lifecycle. | 21:50 |
| clarkb | jog0: I get that, but hacking feels like the same type of upstream as pep8 | 21:50 |
| mordred | notmyname: it's because we need a running cloud | 21:50 |
| *** dkliban is now known as dkliban_afk | 21:50 | |
| mordred | notmyname: and the only place we currently have to get one of those is by running devtack | 21:50 |
| sdague | clarkb: I think the difference is that if we only release on boundaries, you'll be getting more code copy / pasted back into the projects | 21:51 |
| sdague | so they can get extra checks | 21:51 |
| mordred | notmyname: unfortunatley, openstack+docker isn't likely to help, because we'd still need to run devstack in the docker | 21:51 |
| mordred | and _getting_ a new fresh machine for us takes 0 seconds | 21:51 |
| notmyname | mordred: ya, just thinking about preseeding stuff | 21:51 |
| mordred | since we pre-spin them in the pool | 21:51 |
| mordred | we already do that :) | 21:51 |
| sdague | maybe if hacking had new checks off by default until a release boundary and there was a way to opt in | 21:52 |
| clarkb | sdague: how is that different than pinning pep8 though? | 21:52 |
| *** sarob has quit IRC | 21:52 | |
| clarkb | pep8 releases often and can provide more checks, but we have collectively agreed to follow a consistent set of rules for six months | 21:53 |
| sdague | clarkb: because we're writing the hacking rules during a release cycle based on the review feedback we constantly need to give people | 21:53 |
| sdague | to actively reduce review overhead | 21:53 |
| *** sarob_ has quit IRC | 21:53 | |
| mordred | but our new checks are written specifically in response to things the project is doing | 21:53 |
| jog0 | clarkb: what they said | 21:54 |
| *** dina_belova has joined #openstack-infra | 21:54 | |
| clarkb | fungi: https://review.openstack.org/#/c/37461/ weren't you involved in that situation? do you know if we can remove that pin? | 21:55 |
| notmyname | mordred: odd dependency tree in https://review.openstack.org/#/c/40120/. is the migration to pbr required for that patch? | 21:55 |
| mordred | notmyname: it is not. I can rebase that if you'd like | 21:56 |
| *** dolphm has quit IRC | 21:56 | |
| notmyname | mordred: it would allow it to get merged ;-) | 21:56 |
| clarkb | notmyname: mordred did that for every pbr migration and it was a pain point :) | 21:56 |
| mordred | notmyname: it was on top of the pbr patch because of the following patch which is about working on the autosyncing stuff | 21:56 |
| *** vipul-away is now known as vipul | 21:56 | |
| mordred | notmyname: we're about to be doing auto-updating from reuqirements in the devstack gate, which will have issues if you're not on pbr ... so I'd love to re-visit the pbr patch at your convenience | 21:57 |
| notmyname | so we're getting to the point of having computers write our code for us now? ;-) | 21:58 |
| *** dina_belova has quit IRC | 21:59 | |
| clarkb | notmyname: I think jenkins is the number one contributor if you go by lines changed :) | 21:59 |
| notmyname | which we all know is the right way to measure progress | 22:00 |
| clarkb | all of the stats people neglect to give the infra team credit for that :) | 22:00 |
| *** vijendar has quit IRC | 22:00 | |
| notmyname | (the stats numbers that are published are somewhat of an annoyance to me) | 22:00 |
| clarkb | and to be fair Jenkins is just applying translation updates. the translators should be getting the credit | 22:01 |
| jeblair | we could forge that if we had the info... | 22:01 |
| jeblair | mordred: and actually, we have the info for reqs changes... | 22:01 |
| notmyname | mordred: your commit message says you added copyright headers. you patch added license headers. shall I kill the merge process? | 22:02 |
| jeblair | mordred: you should set the git author to be == to os/requirements git author | 22:02 |
| jeblair | oh, but i bet you amend old commits, don't you? that gets to be harder. | 22:02 |
| *** burt has quit IRC | 22:04 | |
| mordred | notmyname: it is a copyright license header ... so I don't think that copyright header is egregiously incorrect - but I will re-spin if you feel differently | 22:04 |
| notmyname | mordred: seems that there is obviously not a copyright line in any of those files. obviously I'm fine with the change (my initial +2/+1), but the commit message seems confusing for when someone (ie me) has to go back through history to see what happened | 22:06 |
| clarkb | mgagne: is there a way to tell puppet lint to ignore errors on a specific line? | 22:06 |
| clarkb | pabelanger: ^ | 22:06 |
| *** sarob has joined #openstack-infra | 22:06 | |
| mgagne | clarkb: not that I'm aware of. I guess you are looking for pep8 like flags | 22:07 |
| clarkb | mgagne: ya, | 22:07 |
| clarkb | er yes, let me give you a link to the thing I have a problem with | 22:07 |
| mgagne | clarkb: funny, I'm working on fixing (again) puppet-lint for our modules | 22:07 |
| clarkb | mgagne: the namevar for a resource can be exposed through $name right? | 22:08 |
| notmyname | mordred: to be explicit, yes I would like to see a commit message that is clearer for when we need to do `git blame` or `git shortlog` six months from now | 22:08 |
| mgagne | clarkb: yes, and $title in some circumstances | 22:08 |
| clarkb | mgagne: the inline comment on https://review.openstack.org/#/c/37794/ | 22:09 |
| clarkb | mgagne: your feedback would be valuable | 22:09 |
| mgagne | clarkb: oh god. I'll check | 22:09 |
| clarkb | pleia2: ^ | 22:09 |
| openstackgerrit | Steve Baker proposed a change to openstack-infra/config: Do not run python jobs for tripleo-image-elements https://review.openstack.org/39858 | 22:10 |
| *** mrodden has quit IRC | 22:10 | |
| mordred | notmyname: great! on it | 22:11 |
| mordred | notmyname: patches incoming | 22:11 |
| *** nijaba_ has quit IRC | 22:11 | |
| *** nijaba has joined #openstack-infra | 22:12 | |
| *** nijaba has quit IRC | 22:12 | |
| *** nijaba has joined #openstack-infra | 22:12 | |
| mgagne | clarkb: check is looking for "${" | 22:13 |
| mgagne | clarkb: https://github.com/sbryant/puppet-lint/blob/master/lib/puppet-lint/plugins/check_strings.rb#L64 | 22:13 |
| mgagne | clarkb: but removing {} would trigger "variable not enclosed in {}" | 22:14 |
| mordred | notmyname: also - I mentioned a gate automation need to escalate our conversation about pbr - I just verified that that is incorrect | 22:14 |
| mordred | notmyname: I still am ready to re-have that conversation - but I have the appropriate safeguards in place for the other things around requirements processing to work | 22:14 |
| mordred | jeblair: AH - my brain JUST NOW was able to read your text above | 22:15 |
| clarkb | mgagne: and single quoting the string will fail because there is a varialbe in an uninterpoaltable string? | 22:15 |
| mgagne | clarkb: yes | 22:15 |
| mordred | jeblair: I am re-committing, but I'm pulling based on topic | 22:15 |
| mgagne | clarkb: trying something on my side: would escaping { with \ do the job? | 22:15 |
| clarkb | mgagne: I think if we can single quote that string we will be fine which is why I was looking for a way to annotate the line to ignore lint errors | 22:15 |
| mordred | jeblair: so I could add in co-authored-by lines | 22:16 |
| clarkb | mgagne: oh maybe | 22:16 |
| mordred | jeblair: OR - I could not re-commit | 22:16 |
| mgagne | clarkb: puppet-lint isn't that fancy sorry ^^' | 22:16 |
| mordred | jeblair: I could commit, forging author information, and commit on top of the last one | 22:16 |
| mordred | jeblair: so that they depend on each other | 22:16 |
| jeblair | mordred: co-authored-by might be a good way of compromising between attribution and not annoying people with multiple auto commits. | 22:16 |
| mordred | jeblair: but each have the author of the change | 22:16 |
| mordred | jeblair: ++ | 22:16 |
| mordred | jeblair: mind if we file that as a feature enhancement for the future? | 22:17 |
| jeblair | mordred: (perhaps if they are merged quickly in practice without recommits, we could back down later) | 22:17 |
| clarkb | mordred: https://review.openstack.org/#/c/37987/4/modules/openstack_project/files/zuul/layout.yaml does that play nice with the eventual state of the requirements world? | 22:17 |
| jeblair | mordred: sure | 22:17 |
| mordred | sdague: also- when you said you wanted to hold off on auto-update patches ... | 22:17 |
| mordred | clarkb: yes. the -requirements jobs will not change | 22:17 |
| clarkb | mordred: I have a hunch we may end up breaking savanna if things in the requirements world move away from them | 22:17 |
| clarkb | mordred: ok, only devstack and friends change? | 22:18 |
| mordred | clarkb: I was musing earlier that we sohuld make the auto-requirements patch creation an opt-in process for projects like savanna that might want that | 22:18 |
| jeblair | I'm starting a graceful shutdown of jenkins01 and jenkins02 to work on the credential store thing. | 22:18 |
| mordred | also - futher feature request | 22:19 |
| clarkb | jeblair: have fun | 22:19 |
| clarkb | jeblair: out of curiousity are we running latest jenkins release? | 22:19 |
| clarkb | or are you using the same jenkins verson across the baord? | 22:19 |
| jeblair | clarkb: 01 and 02 are; jenkins.o.o is not | 22:19 |
| clarkb | cool | 22:19 |
| mordred | jeblair: I saw puppet patches forplugins come across - are you using those for these guys? | 22:21 |
| mgagne | clarkb: it does not work | 22:21 |
| clarkb | mgagne: :( | 22:21 |
| mgagne | clarkb: I end up with "test $\{name\} test" in my file | 22:21 |
| clarkb | mgagne: what if $\{name\} is in a double quoted string? | 22:22 |
| jeblair | mordred: yes, i did use the patch i proposed. | 22:22 |
| clarkb | of course that will also fail lint because no variable will be in that string | 22:22 |
| mgagne | clarkb: I'll try | 22:22 |
| mordred | jeblair: excellent. I thnk that's sexy | 22:22 |
| jeblair | mordred: bodepd gave it to us. it's cool. :) | 22:22 |
| *** pentameter has quit IRC | 22:22 | |
| clarkb | anyone else want to review https://review.openstack.org/#/c/38119/4 before I approve it? It adds some python3 compatibilty to jeepyb and bumps the hacking version | 22:23 |
| clarkb | mordred: https://review.openstack.org/#/c/38176/ is that going to be fixed? | 22:24 |
| mgagne | clarkb: WARNING: double quoted string containing no variables on line 2 | 22:24 |
| mordred | clarkb: yes. right noiw | 22:25 |
| clarkb | mgagne: I expected that. This makes me sad | 22:25 |
| mgagne | clarkb: I'm a sad panda too now | 22:25 |
| mgagne | clarkb: I'll try to see if I can hack something | 22:25 |
| *** prad_ has quit IRC | 22:26 | |
| openstackgerrit | Monty Taylor proposed a change to openstack-infra/config: Gate jeepyb and openstack/requirements on pbr https://review.openstack.org/38176 | 22:28 |
| mordred | clarkb: ^^ | 22:28 |
| mordred | there you go | 22:28 |
| mordred | you know what - once we have that - our requirements are going to have the HELL tested out of them | 22:29 |
| *** datsun180b has quit IRC | 22:30 | |
| clarkb | I am -2'ing changes like https://review.openstack.org/#/c/38114/ since global-requirements.txt makes it a noop | 22:33 |
| mgagne | clarkb: I think I had the worst idea ever | 22:33 |
| clarkb | mgagne: I am ready for it | 22:33 |
| mgagne | clarkb: testing atm | 22:33 |
| mordred | I think I believe there is enough testing on requirements now that I'm going to start reviewing them | 22:34 |
| mordred | too | 22:34 |
| mordred | clarkb: how about just abandoning those | 22:34 |
| clarkb | mordred: I can do that too | 22:35 |
| mgagne | clarkb: chomp("file:///var/lib/git/\${name}.git\n") | 22:35 |
| clarkb | ahahahahahah sorry | 22:36 |
| mgagne | clarkb: there's an exception for \n in the check | 22:36 |
| clarkb | that is glorious | 22:36 |
| clarkb | mgagne: can you reply to my comment on the change with that info? | 22:36 |
| mgagne | clarkb: sure | 22:36 |
| clarkb | awesome, thank you for looking at this | 22:37 |
| mgagne | clarkb: np | 22:37 |
| clarkb | mordred: abandoned | 22:39 |
| mordred | woot | 22:39 |
| openstackgerrit | Monty Taylor proposed a change to openstack/requirements: Bump pbr requirment to 0.5.21 https://review.openstack.org/40117 | 22:39 |
| clarkb | mordred: will git handle the file rename gracefully or will everyone need to rebase? I think rebasing is necessary | 22:40 |
| mordred | clarkb: what file rename? | 22:40 |
| *** krtaylor has joined #openstack-infra | 22:40 | |
| mordred | ah. hrm. anyone with a patch to test-requirements will need to rebase | 22:41 |
| mordred | requirements.txt shoudl be fine | 22:41 |
| clarkb | mordred: I can leave comments on all of those changes in a bit | 22:41 |
| clarkb | mordred: https://review.openstack.org/#/c/38176/4/modules/openstack_project/files/jenkins_job_builder/config/requirements.yaml does conflict with the savanna zuul layout change | 22:42 |
| mordred | what? | 22:42 |
| clarkb | mordred: you are removing the -install job | 22:42 |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/devstack-gate: Use Jenkins credentials store if specified https://review.openstack.org/40310 | 22:42 |
| clarkb | oh, I am mixingthe jobs up again | 22:42 |
| clarkb | ignore me | 22:43 |
| mordred | :) | 22:43 |
| jeblair | clarkb, mordred: can you aprv https://review.openstack.org/40310 ? | 22:43 |
| mordred | on it | 22:43 |
| jeblair | i plan to handle the migration by stopping puppet on all the devstack-launch nodes | 22:43 |
| jeblair | and manually applying the change i'm about to propose only to the new ones | 22:43 |
| jeblair | that should allow old devstack-launch to work as it has been, while updating the new ones with the new behavior | 22:44 |
| jeblair | i have tested that manually on devstack-launch01 (with jenkins01) | 22:44 |
| clarkb | reviewing | 22:44 |
| *** emagana_ has quit IRC | 22:45 | |
| clarkb | jeblair: then we will let devstack-launch.slave die on the vine? | 22:45 |
| jeblair | clarkb: yep | 22:45 |
| jeblair | clarkb: i anticipate that happening very soon, so i'm not too worried about just leaving puppet off on that node | 22:45 |
| clarkb | wfm | 22:45 |
| clarkb | approved | 22:46 |
| jeblair | clarkb: actually, i think we can even keep puppet going everywhere (now that i'm writing the config change) | 22:48 |
| clarkb | jeblair: don't apply the credentials change to the config on the old host? | 22:48 |
| jeblair | yeah | 22:49 |
| clarkb | mordred: fungi https://review.openstack.org/#/c/40301/ will make apevec happy | 22:50 |
| clarkb | I will approve 40301 in a bit to make sure it gets in before the next round of periodic jobs | 22:50 |
| openstackgerrit | A change was merged to openstack-infra/jeepyb: Allow hacking 0.6.0 and fix versions https://review.openstack.org/38119 | 22:51 |
| jeblair | clarkb, mordred: we need to work on getting my puppet changes merged | 22:51 |
| jeblair | so that we can allow jjb to run on the new jenkins nodes | 22:51 |
| mordred | jeblair: I will now do that | 22:51 |
| jeblair | mordred, clarkb: https://review.openstack.org/#/c/40041/ is the tip | 22:52 |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/config: Use jenkins credentials store with d-g https://review.openstack.org/40312 | 22:53 |
| jeblair | plus that ^ | 22:53 |
| clarkb | ok reviewing | 22:54 |
| *** dina_belova has joined #openstack-infra | 22:55 | |
| mordred | jeblair: chain +2 by me | 22:55 |
| mordred | there are a couple fo small things people have brought up - but I'm fine with addressing those in a follow on | 22:55 |
| clarkb | jeblair: should I be picky about the plugins? eg do we need the bazaar plugin? | 22:55 |
| jeblair | clarkb: propose a change to remove them? | 22:56 |
| jeblair | (which won't remove them, but hey, they're already installed so...) | 22:56 |
| clarkb | jeblair: ok we can clean up later like mordred suggested | 22:56 |
| mordred | yeah, something tells me we're going to tinker with this for a while | 22:56 |
| jeblair | clarkb, mordred: i'm replying to the comments. | 22:57 |
| *** changbl has joined #openstack-infra | 22:58 | |
| mordred | jeblair: ok. I believe I have now reviewed all of your patches | 22:59 |
| *** dina_belova has quit IRC | 22:59 | |
| openstackgerrit | A change was merged to openstack-infra/config: Remove ZUUL_BRANCH peridic job parameter https://review.openstack.org/40301 | 23:00 |
| clarkb | jeblair: ok I am approving that chain | 23:01 |
| clarkb | and done | 23:01 |
| jeblair | i responded to all the comments; i do think there are things to fixup, but we can hit them in new patches. | 23:01 |
| clarkb | mordred: why are we pinning things like test resources? https://review.openstack.org/#/c/37911/1/test-requirements.txt | 23:02 |
| *** beagles has quit IRC | 23:02 | |
| mordred | clarkb: to give a minimum lower bound | 23:03 |
| *** kpepple_ has quit IRC | 23:03 | |
| *** kpepple has joined #openstack-infra | 23:03 | |
| openstackgerrit | A change was merged to openstack-infra/config: Add jenkins::plugin defined resource type https://review.openstack.org/39615 | 23:03 |
| clarkb | mordred: but why? | 23:03 |
| openstackgerrit | A change was merged to openstack-infra/config: Don't automatically restart jenkins on plugin changes https://review.openstack.org/40040 | 23:03 |
| mordred | also - that was the point of that patch, to add lower bounds to anything with an aupper bound to make parsing things easier | 23:03 |
| *** wayneeseguin has quit IRC | 23:03 | |
| openstackgerrit | A change was merged to openstack-infra/config: Add current jenkins plugins https://review.openstack.org/40041 | 23:03 |
| *** jcooley has quit IRC | 23:03 | |
| mordred | clarkb: you _know_ that if there is a comma list, the first will be power | 23:03 |
| clarkb | mordred: the discussion we had after the broken xattr stuff implied we want to know when upstream breaks us | 23:03 |
| mordred | lower | 23:03 |
| mordred | I duno | 23:04 |
| clarkb | giving a lower bound to have a lower bound prevents that | 23:04 |
| mordred | why? | 23:04 |
| jeblair | mordred: your devstack change is in danger of failing due to a failed (flaky?) neutron test. | 23:04 |
| mordred | jeblair: wow | 23:04 |
| mordred | it's been so long since that's been the case | 23:04 |
| clarkb | mordred: because if testresources 0.4 or whatever comes out we will never use it | 23:04 |
| *** sarob has quit IRC | 23:04 | |
| mordred | clarkb: you are talking about upper bounds | 23:05 |
| clarkb | mordred: er ya | 23:05 |
| clarkb | s/pinning/setting an upper bound/ | 23:05 |
| *** wayneeseguin has joined #openstack-infra | 23:05 | |
| *** sarob has joined #openstack-infra | 23:05 | |
| openstackgerrit | Mathieu Gagné proposed a change to openstack-infra/config: Fix puppet-lint for bare puppet modules https://review.openstack.org/40314 | 23:05 |
| mordred | clarkb: I believe that we, in general, have decided that we _do_ want to protect against breaking changes from upstream at times by pinning in requirements | 23:05 |
| mordred | that's different than protecting by rejecting things from the mirror in the gate | 23:06 |
| *** jcooley has joined #openstack-infra | 23:06 | |
| clarkb | we are not rejecting things from the mirror in the gate | 23:06 |
| mordred | I know, because we do not want to | 23:06 |
| clarkb | I feel like every time we discuss this I come away with a different impression of what people want | 23:06 |
| mordred | it's differnet issues | 23:06 |
| *** beagles has joined #openstack-infra | 23:06 | |
| clarkb | mordred: what is the issue here? | 23:06 |
| jeblair | clarkb: the upper bounds are supposed to be set such that they will include stable or security-style updates, but not breaking api changes. | 23:06 |
| mordred | we want what is expressed in the file to be somethign that will work in reality outside of our systems | 23:07 |
| jeblair | clarkb: that's vague, up to interpretation, and varies by project policy for every upstream project. | 23:07 |
| clarkb | I feel like that particular pin was pulled out of a hat randomly | 23:07 |
| mordred | so putting a bound there is fine, because someone downloading nova from the internet and installing it will get the set of software described | 23:07 |
| clarkb | lifeless is really good about backward compat | 23:07 |
| clarkb | so I think what I am hearing is the in the general case we may want to pin upper bounds as necessary | 23:08 |
| mordred | yes | 23:08 |
| mordred | although, we tend to be mildly agressive | 23:08 |
| clarkb | so now I am focusing on that particular pin. Any idea why? | 23:08 |
| mordred | no | 23:08 |
| mordred | I do not have strong feelings about that pin | 23:08 |
| lifeless | clarkb: I'm good but not perfect :) | 23:08 |
| clarkb | I do because zigo and others tried pulling this before... and I told them all no because it was busy work that didn't get us anyhting | 23:09 |
| mordred | jeblair: I also got a lost on a grenade tests | 23:09 |
| clarkb | I think some people feel that they must package the upper bounds.... | 23:09 |
| clarkb | rather than understand the range is acceptable | 23:09 |
| *** sarob has quit IRC | 23:09 | |
| jeblair | mordred: "gate-grenade-devstack-vm: SUCCESS" ? | 23:10 |
| fungi | clarkb: yeah, https://review.openstack.org/37461 is the follow up to requirements i *should* have proposed but forgot. it was in there to support nova, removed by my https://review.openstack.org/38012 change | 23:10 |
| jeblair | mordred: for 35705,16 | 23:10 |
| mordred | jeblair: this was for a keystoneclient | 23:10 |
| *** nijaba has quit IRC | 23:10 | |
| * fungi is scrollbacking for a bit, looks like | 23:10 | |
| mordred | jeblair: https://review.openstack.org/#/c/40284/ | 23:10 |
| mordred | jeblair: http://logs.openstack.org/84/40284/2/check/gate-grenade-devstack-vm/20f630b : LOST in 12m 02s | 23:11 |
| *** nijaba has joined #openstack-infra | 23:12 | |
| clarkb | mordred: see comments on https://review.openstack.org/#/c/39967/ | 23:14 |
| clarkb | fungi: I have approved 37461, we'll have to see how it does in the gate with all of the new shinyness | 23:16 |
| jeblair | mordred: that node was re-used because the first job was aborted while waiting for the devstack-inprogress job | 23:17 |
| *** rcleere has quit IRC | 23:18 | |
| mordred | clarkb: only need from future importprint_function when you're going to use one of the new print function features | 23:18 |
| jeblair | mordred: which eventually finished after the second job started; and likewise, the complete job from the first job also finished, and then the node was deleted while the second job was running. | 23:18 |
| jeblair | clarkb: i believe i'm at the point where we could merge your change that tests for idleness before deleting a node | 23:19 |
| clarkb | mordred: does it not depend on order of import? I am too lazy to go around and figure out what the order should be | 23:19 |
| clarkb | jeblair: oh, is that still a good thing? I have abandoned it and need to remove the race condition | 23:19 |
| clarkb | jeblair: but I can do that if you want | 23:19 |
| mordred | clarkb: nope | 23:19 |
| jeblair | clarkb: if it needs work, it may not be worth it... | 23:19 |
| clarkb | jeblair: its not much work, I have to set the node to offline before checking if it is idle | 23:19 |
| clarkb | jeblair: one extra line basically | 23:20 |
| jeblair | oh we don't want to set the node to offline | 23:20 |
| jeblair | that confuses the hell out of people | 23:20 |
| clarkb | oh | 23:20 |
| clarkb | but it is a safe thing to do | 23:20 |
| clarkb | and removes the idle check race condition | 23:20 |
| *** vipul is now known as vipul-away | 23:20 | |
| jeblair | we'll see lots of people in here saying "hey i got an error because the node is offline" | 23:20 |
| clarkb | :/ | 23:20 |
| jeblair | clarkb: we used to do that, before doing the relabel | 23:20 |
| jeblair | clarkb: the other thing is that even that is still subject to errors | 23:21 |
| clarkb | does jenkins ignore the offline state at times? | 23:21 |
| jeblair | clarkb: since the result of your change is that we allow jobs to run on nodes that have already been used (but not had their labels updated)... | 23:22 |
| clarkb | which change? | 23:22 |
| stevebaker | clarkb, jeblair, any chance of another look at https://review.openstack.org/#/c/39858/ ? | 23:22 |
| jeblair | clarkb: with the current runtime for the label update jobs being so high, it's possible that the first job might actually get far enough to affect the second. | 23:23 |
| jeblair | clarkb: the change under discussion. :) | 23:23 |
| clarkb | jeblair: it doesn't allow that to happen. It tries to reduce the size of the crater if it does happen | 23:23 |
| jeblair | clarkb: so it's basically substituting one error for another, hopefully with better odds | 23:23 |
| clarkb | jeblair: and log the case | 23:23 |
| jeblair | clarkb: if it does not allow job to run on a node that has already been used (but just not had the label updated) then we should not merge it because that's the problem that we're seeing. | 23:24 |
| clarkb | stevebaker: yup looking now. I feel bad ps4 appears to be a result of my bad suggestion in ps2 | 23:24 |
| stevebaker | heh, now we know | 23:24 |
| jeblair | clarkb: at any rate, the work i'm starting next will hopefully eliminate this (and other) problems | 23:25 |
| clarkb | jeblair: I get that it isn't the best fix for a problem, but it is a fix that we could put in now... I think that was what I was going for. I understand we can do it better iwth a lot of work and in the meantime we could use something like that to avoid large explosions | 23:26 |
| jeblair | clarkb: i agree, which is why i said i'm okay merging it, before you indicated that it needed more work. | 23:27 |
| jeblair | clarkb: also you indicated it did not solve the problem at hand. | 23:27 |
| *** sandywalsh has quit IRC | 23:27 | |
| clarkb | correct I don't think it solves all instances of run a job twice on the node | 23:27 |
| *** pcrews has quit IRC | 23:27 | |
| clarkb | it solves the we killed the node while your test was running problem | 23:27 |
| jeblair | which i believe only happens if a node is re-used. and i believe the only time that is currently happening is because the first job is aborted while waiting on the label job. | 23:28 |
| clarkb | gotcha | 23:28 |
| jog0 | sdague: can you take a look at https://review.openstack.org/#/c/39140/ | 23:28 |
| jog0 | sdague: its the blocker for hacking 0.7 | 23:29 |
| jog0 | clarkb: FYI hacking 0.7 is a bug fix release that fixes a broken test | 23:29 |
| jeblair | stevebaker, lifeless: tie can build an sdist but not run pep8? | 23:30 |
| stevebaker | there is no python, but it is packaged with pbr | 23:30 |
| clarkb | reed: https://review.openstack.org/#/c/38761/ | 23:31 |
| bodepd | jeblair: surprised to see you finish that patch off | 23:31 |
| bodepd | jeblair: but thanks :) | 23:32 |
| jeblair | bodepd: i had an immediate use for it. it was timely. thank you. :) | 23:32 |
| reed | clarkb, checking | 23:32 |
| reed | clarkb, can't approve it | 23:33 |
| clarkb | jeblair: and by adding multiple launch slaves and reducing the total pool per launch slave we should see the instance of that decrease? | 23:33 |
| lifeless | jeblair: thats correct | 23:33 |
| clarkb | reed: I know, but I can approve it if you give it a +1 | 23:33 |
| reed | done | 23:33 |
| lifeless | jeblair: it has no python code in it; it's using setup.py to build the distribution just because. | 23:33 |
| clarkb | reed: thanks | 23:33 |
| jeblair | clarkb: i hope that jenkins will be more responsive under less load, while i work on the d-g updates. | 23:33 |
| jeblair | clarkb: (we may not want to increase our slave count too much while that's in progress) | 23:33 |
| *** wu_wenxiang has joined #openstack-infra | 23:34 | |
| clarkb | ++ | 23:34 |
| *** danger_fo is now known as danger_fo_away | 23:34 | |
| jeblair | lifeless: do the shell scripts themselves get packaged? if not, what does? | 23:34 |
| openstackgerrit | A change was merged to openstack-infra/config: Do not run python jobs for tripleo-image-elements https://review.openstack.org/39858 | 23:34 |
| stevebaker | thanks, now we need to do the same for disimage-builder ;) | 23:35 |
| openstackgerrit | A change was merged to openstack-infra/config: Use jenkins credentials store with d-g https://review.openstack.org/40312 | 23:35 |
| lifeless | jeblair: they do, by the setup.py/setup.cfg stuff | 23:35 |
| *** vipul-away is now known as vipul | 23:36 | |
| jeblair | lifeless: ok. thanks. fascinating. | 23:36 |
| *** mrodden has joined #openstack-infra | 23:36 | |
| openstackgerrit | A change was merged to openstack-infra/config: Add french list openstack-fr https://review.openstack.org/38761 | 23:37 |
| lifeless | jeblair: we're happy to do it differently if that will help | 23:38 |
| clarkb | fungi: you still around? https://review.openstack.org/#/c/39992/1 has the IPV6 var been added to the env files? | 23:38 |
| lifeless | jeblair: main reason to use setup.py was conformity - avoinding a new toolchain in the project. | 23:38 |
| clarkb | I will aprove 39992 if it has | 23:38 |
| jeblair | lifeless: i have no alternate suggestions. i really do find it an interesting use of the python packaging and distribution system. | 23:39 |
| clarkb | fungi: also you may want to get reservations at shiros if you can. I am not sure what their wait list is like | 23:41 |
| stevebaker | admittedly it is an expedient way of adding dib and tie as installed dependencies in devstack | 23:42 |
| *** sandywalsh has joined #openstack-infra | 23:42 | |
| jeblair | it does seem to have that going for it. if you're pip installing 500 other things, what's one or two more? :) | 23:42 |
| bodepd | jeblair: glad to help! | 23:43 |
| stevebaker | indeed | 23:43 |
| clarkb | bodepd: https://review.openstack.org/#/c/39611/2 makes me sad | 23:44 |
| clarkb | bodepd: any idea why it is blowing up like that? | 23:44 |
| clarkb | bodepd: and any chance you can paste the traceback in a place where formating is preserved? | 23:44 |
| *** jrex_laptop has joined #openstack-infra | 23:46 | |
| mgagne | Are launchpad teams and gerrit groups supposed to be synced? | 23:46 |
| clarkb | mgagne: not anymore | 23:46 |
| mgagne | clarkb: should I want to add a new core reviewers, this would have to be done in gerrit right? | 23:46 |
| clarkb | mgagne: there were when we used lp to manage the CLA but we stopped doing that in favor of the CLA stuff in gerrit and now all gerrit groups are managed through gerrit | 23:46 |
| clarkb | mgagne: yup | 23:46 |
| mgagne | clarkb: thanks for the info | 23:46 |
| bodepd | clarkb: no idea | 23:47 |
| bodepd | clarkb: I just replaced it with package { pip } and moved on | 23:48 |
| bodepd | jenkins is hard to install :( | 23:48 |
| clarkb | mordred: https://review.openstack.org/#/c/39207/ what is with putting pbr on the exclusion list? Don't you forcefully update setup.py? | 23:48 |
| clarkb | bodepd: we use jenkins' .debs | 23:48 |
| bodepd | clarkb: I unfortunately have xml templates | 23:49 |
| *** Shrews has quit IRC | 23:49 | |
| bodepd | clarkb: which makes me terrified to change any versions | 23:49 |
| bodepd | clarkb: I actually hate jenkins b/c of its crappy API | 23:50 |
| bodepd | clarkb: can we just move to build bot? (if it's any better...) | 23:50 |
| clarkb | bodepd: ask mordred. I believethe answer is "NO!" | 23:50 |
| jeblair | bodepd: zuul doesn't depend on jenkins, so you could hook anything up to gearman to run your jobs. | 23:51 |
| bodepd | I was thinking about that today | 23:51 |
| bodepd | I'm going to defer to what-ever you guys do for now | 23:52 |
| jeblair | bodepd: i think we may start doing that for some jobs eventually (as are the wikipedia folks). but the hard part is having a system that both runs untrusted random code submitted by anyone on the internet, and also builds your release artifacts. | 23:52 |
| jeblair | so, um, we'll be thinking about it carefully. :) | 23:53 |
| *** shardy has quit IRC | 23:54 | |
| bodepd | yeah | 23:55 |
| bodepd | code review? | 23:55 |
| *** dina_belova has joined #openstack-infra | 23:55 | |
| bodepd | require a +1 from core before running CI is a pain, but the solution to that problem... | 23:55 |
| clarkb | bodepd: for the pip failure, I think that may be related to pbr, setuptools, and distribute madness | 23:55 |
| jeblair | we run jobs on patchset uploaded (by popular demand; that was our initial stance) | 23:55 |
| clarkb | bodepd: is your package {} ensuring latest? | 23:56 |
| bodepd | no | 23:56 |
| bodepd | wait, let me check... | 23:56 |
| jeblair | bodepd: also, obfuscated code would probably slip by. | 23:56 |
| bodepd | https://gist.github.com/bodepd/6160748 | 23:56 |
| bodepd | jeblair: openstack itself is much more at risk for that IMO | 23:57 |
| fungi | clarkb: on 39992, yes that var was added to all the shell config snippets on the master (i think i left a review comment to that effect in it when i uploaded, but need to finish catching up on scrollback before i go hunting) | 23:57 |
| clarkb | bodepd: hmm that is essentially what the pip module is doing, this is very odd | 23:57 |
| bodepd | clarkb: yeah, I didn't understadn it either. | 23:57 |
| bodepd | the pip package installs a few other packages | 23:58 |
| clarkb | fungi: you did, I see it now | 23:58 |
| bodepd | maybe the version of those is no good for zuul? | 23:58 |
| clarkb | bodepd: it installs things like python all dev so that you have python headers if pip installed things need them | 23:58 |
| bodepd | I can recreate | 23:58 |
| bodepd | it'll take about 10 minutes | 23:58 |
| clarkb | bodepd: its possible. I am thinking it is related to the setuptools madness | 23:58 |
| fungi | clarkb: adding shiros to the hit list, thanks! | 23:58 |
| bodepd | is there something I should try? | 23:58 |
| bodepd | clarkb: ^^^ like removing setuptools from the pip class? | 23:59 |
| clarkb | bodepd: if you recreate try uninstalling python-pkg-resources python-setuptools then install python-pip and pip install -U setuptools | 23:59 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!