Tuesday, 2014-09-30

*** armax has quit IRC00:01
*** hdd has quit IRC00:02
fungii love how the operators list explodes with malcontent each time we eol a stable branch00:02
mordredhehe00:03
*** wenlock has quit IRC00:04
*** baoli has quit IRC00:05
*** tomoe has left #openstack-infra00:07
*** todd_dsm has joined #openstack-infra00:08
clarkbfungi has that happened already? I am not seeing it if so00:08
*** tgohad has joined #openstack-infra00:09
*** JayJ has joined #openstack-infra00:10
fungiclarkb: http://lists.openstack.org/pipermail/openstack-operators/2014-September/005229.html00:10
krotscheckHrmp. That didn’t work. Ahh well, I’ll poke it more in the morning.00:11
sdaguejogo: so you caught a real bug, but not the one you think :)00:11
*** tsg has quit IRC00:11
*** baoli has joined #openstack-infra00:12
*** yamamoto has joined #openstack-infra00:13
openstackgerritSean Dague proposed a change to openstack-infra/elastic-recheck: fix Launchpad link name  https://review.openstack.org/12492800:13
sdaguejogo: the link in ER that I'm adding isn't to launchpad, it's to that graph on the page00:13
sdagueso http://status.openstack.org/elastic-recheck/#1334109 will take you to that bug entry in the er page00:15
jlkit's almost as if people don't understand that if they want long term maintenance of a software set, they should get said software from a long term maintenance vendor00:15
jlklike, say Red Hat00:15
*** tgohad has quit IRC00:17
fungijlk: well, we'd love it if red hat, ubuntu, suse and others dedicated more resources to maintaining our stable branches upstream longer, but the unfortunate truth is that after a time it's too much work and things are far simpler for them to just keep it working with whatever software dependencies they shipped with it when it released00:17
jlksure, it's on them to keep the old stuff working within their offering00:18
jlkwhere they have more control over the userland00:18
jlkbut end users can't expect that to come for free00:18
*** SumitNaiksatam has quit IRC00:18
fungijlk: i'm actually thrilled that they and others manage to keep support for our releases updated and fresh for as much as a year, and take care of more than one in parallel even00:19
*** xchu has joined #openstack-infra00:19
*** marcoemorais has quit IRC00:19
jlkyeah, as you stated in the thread, that's well beyond what most upstreams can manage00:19
jlkmost are HEAD or GTFO00:19
fungisupporting stable branches upstream for longer is mainly just a question of resources, and so far people only manage to step up and keep it going for roughly a year per release00:20
*** otherwiseguy has joined #openstack-infra00:20
*** bdpayne has quit IRC00:21
*** zz_dimtruck is now known as dimtruck00:21
fungibut it's not like we arbitrarily choose to end support for releases, and i find it amusing that there are so many people who just don't get that00:21
krotscheckA question about jjb- when I set the flag of ‘keep hierarchy’ to false, does it flatten everything (the uploaded files and the path in the project directory) or does it only flatten out the latter?00:21
*** marcoemorais has joined #openstack-infra00:21
*** bhuvan has quit IRC00:23
mordredkrotscheck: yes00:23
krotscheckmordred: I’m confused.00:24
krotscheckmordred: I need my draft config file to be in the same directory as the js build output, and right now it’s putting it into ~/test/draft/. I was hoping if I fiddled the keep-hierarchy bit it’d just dump all the files into ~/00:25
krotscheckBut if that’s not the case I need to figure something else out00:25
*** baoli has quit IRC00:26
*** praneshp has quit IRC00:26
*** andreaf has quit IRC00:26
*** marcoemorais has quit IRC00:27
*** andreaf has joined #openstack-infra00:27
*** marcoemorais has joined #openstack-infra00:27
*** yamahata has quit IRC00:28
*** unicell has joined #openstack-infra00:29
*** praneshp has joined #openstack-infra00:30
*** praneshp has quit IRC00:30
mordredhrm00:31
mordredkrotscheck: you have keep-hierarchy set to true though?00:32
*** dims has joined #openstack-infra00:32
mordredkrotscheck: https://review.openstack.org/#/c/124606/2/jenkins/jobs/javascript.yaml00:33
mordredor?00:33
*** baoli has joined #openstack-infra00:33
krotscheckmordred: Yeah, so I’m getting a ‘~/dist/‘ folder with the compiled assets, and a ~/test/draft folder with the config file. Optimally I want them both in the same directory.00:33
krotscheckmordred: http://paste.openstack.org/show/116914/00:34
*** mmaglana has quit IRC00:35
fungikrotscheck: the short and unfortunate answer is that the scp-publisher plugin can either give you a flat dump of all the files, or preserve the original directory structure and root it at a location of your choosing, but there's no good path mangling solution for it00:36
fungikrotscheck: so if possible, your job should arrange the files relative to your git clone working directory exactly how and where you expect to see them under the target directory00:37
*** todd_dsm has quit IRC00:39
krotscheckALright, that should be doable00:39
krotscheckJust not optimal00:39
*** bradjones has quit IRC00:40
*** todd_dsm has joined #openstack-infra00:40
fungiyeah, the ftp publisher plugin can strip leading path parts from the source. a feature i really wish the scp publisher had00:40
fungiyou know, without giving your passwords to anyone who has a router between you and the server00:41
*** marcoemorais has quit IRC00:43
*** gyee has quit IRC00:46
openstackgerritpatrick-crews proposed a change to openstack-infra/elastic-recheck: Adding query namespace to elastic-recheck  https://review.openstack.org/10195800:47
*** bradjones has joined #openstack-infra00:48
*** bradjones has quit IRC00:48
*** bradjones has joined #openstack-infra00:48
mordredfungi: or, we can just replace it all with swift00:48
mordredfungi: OR00:49
mordredfungi: wait for it00:49
mordredfungi: wait for it00:49
mordredfungi: wait for it00:49
mordredfungi: AFS00:49
mordredBOOM00:49
* fungi didn't see that coming at all00:49
*** rwsu has quit IRC00:49
* mordred drops amazing on everyone00:49
*** gokrokve has joined #openstack-infra00:49
*** salv-orlando has quit IRC00:49
*** harlowja_away is now known as harlowja00:49
mordredfungi: we could write a bot that just responds to all questions with "use AFS"00:50
*** gokrokve has quit IRC00:51
*** gokrokve_ has joined #openstack-infra00:51
*** praneshp has joined #openstack-infra00:51
*** chuckC has quit IRC00:53
morganfainbergmordred, lol!00:53
fungimordred: i thought that was already your core routine?00:54
mordredfungi: use AFS!00:54
morganfainbergmordred, so let me get this straight, AFS?00:56
*** esker has joined #openstack-infra00:57
mordredmorganfainberg: use AFS!00:57
*** todd_dsm has quit IRC00:58
openstackgerritA change was merged to openstack-dev/hacking: Stop using intersphinx  https://review.openstack.org/12132801:00
mordredwoot01:00
*** esker has quit IRC01:01
*** marcoemorais has joined #openstack-infra01:01
*** signed8bit has joined #openstack-infra01:02
* nibalizer considers the benefits of an AFS hilight01:02
mordrednibalizer: you know the good topics have come up when AFS gets dropped01:02
nibalizermordred: this? http://www.openafs.org/01:02
mordrednibalizer: yup01:02
* nibalizer wonders if this is like kerberos and has competing impletmenations?01:02
funginibalizer: it relies heavily on kerberos01:03
fungiand yeah, there are a few afs implementations01:03
nibalizeroh look there are openafs packages in ubuntu01:03
mordredyeah. it's been a mainstream thing for a WHILE01:04
funginibalizer: http://en.wikipedia.org/wiki/Andrew_File_System#Implementations01:04
dstufftmordred: I considered using AFS01:04
*** ChuckC_ has quit IRC01:04
nibalizer'a fourth implementation, unnamed, has been randomly shoved into the kernel by RedHat'01:05
mordreddstufft: corvus and I are going to spin up an AFS cell for infra when he gets back from vacation01:05
fungiit'll be just like old times! ;)01:05
mordredfungi: finally01:05
*** andreaf has quit IRC01:05
fungibetween that and the nntp backend for our listserv...01:05
mordredfungi: I'm really not kidding about that ...01:05
* fungi takes back the 90s01:06
dstufftmordred: instead we used glusterfs01:06
*** andreaf has joined #openstack-infra01:06
*** otherwiseguy has quit IRC01:06
dstufftI'm not sure we made the right choice01:06
mordredfungi: we could replace freenode IRC with a LambdaMOO server01:06
mordreddstufft: I think they solve slightly different usecases01:06
fungimordred: tinymush ftw, luser!01:06
mordreddstufft: glusterfs to me seems really great if you do want something that produces NFS mounts01:07
* fungi jokes... i mostly developed extensions to proper mud codebases, after all... mosly diku*01:07
dstufftmordred: glusterfs might be great if you have consitent IO01:08
mordredyes. that too01:08
dstufftIt's not great for anything except frustration if you don't have consistent IO :(01:08
fungidstufft: on the other hand, if you stick it in a cloud...01:08
mordredugh01:08
*** shashankhegde has quit IRC01:09
mordredfungi: how fun would it be if instead of infra we had rooms in a MUD - so that if you came in to infra, you actually had an outer room with a dragon or something01:09
*** unicell has quit IRC01:09
*** wenlock has joined #openstack-infra01:09
mordredfungi: and if we could ACTUALLY throw wet cats at people01:10
mordredI realize I mistyped01:10
mordreds/instead of infra/instead of IRC/01:10
*** otherwiseguy has joined #openstack-infra01:10
* fungi dons his complimentary set of +1 newbie armor01:11
* fungi swings his sword at bug 136790701:11
uvirtbotLaunchpad bug 1367907 in python3.4 "Segfault in gc with cyclic trash" [High,In progress] https://launchpad.net/bugs/136790701:11
* fungi misses, flailing wildly01:11
dstufftmordred: can I add like 12 dragons to participate in distutils-sig01:11
fungidstufft: and train them to eat anyone who wants to be able to silently reupload different package contents with the same version numbers?01:12
dstufftyes01:13
dstufftalso other bad things01:13
fungibecause, you know, if you dictate sane release process to projects, the ones with crappy ideas about how to do releases will flee to github tarball distribution instead, leaving only good software on pypi01:13
fungiand that makes the psf cry01:14
*** gokrokve_ has quit IRC01:14
*** marcoemorais has quit IRC01:15
*** rwsu has joined #openstack-infra01:15
*** gokrokve has joined #openstack-infra01:15
*** SumitNaiksatam has joined #openstack-infra01:15
dstufftfungi: I like the idea that people are fleeing PyPI01:15
clarkbfungi++01:16
dstufftfungi: when in reality, this is what PyPI's usage looks like -> http://d.stufft.io/image/1f0j0k2u1r0Q01:16
fungidstufft: particularly the sorts of people who would get miffed if you told them they could no longer follow user-hostile release practices01:16
dstufft(to be fair, that's downloads not uploads)01:16
*** kgavini has joined #openstack-infra01:17
*** marcoemorais has joined #openstack-infra01:18
dstufftor like http://d.stufft.io/image/1O1K0j0n390s (total requests to PyPI per day)01:19
*** marcoemorais has quit IRC01:19
*** gokrokve has quit IRC01:20
*** marcoemorais has joined #openstack-infra01:20
*** sdake_ has joined #openstack-infra01:20
*** otherwiseguy has quit IRC01:23
*** alexpilotti has quit IRC01:23
*** otherwiseguy has joined #openstack-infra01:23
*** marcoemorais has quit IRC01:24
*** baoli has quit IRC01:24
*** baoli has joined #openstack-infra01:25
*** pcrews has quit IRC01:25
openstackgerritMichael Krotscheck proposed a change to openstack-infra/storyboard-webclient: Added draft build  https://review.openstack.org/12493401:26
*** marcoemorais has joined #openstack-infra01:26
*** signed8bit has quit IRC01:27
*** pcrews has joined #openstack-infra01:27
openstackgerritMichael Krotscheck proposed a change to openstack-infra/project-config: Call build:draft instead of build for js-draft build  https://review.openstack.org/12493601:28
*** yaguang has joined #openstack-infra01:29
*** cyeoh has joined #openstack-infra01:31
*** yamahata has joined #openstack-infra01:32
*** mikedillion has joined #openstack-infra01:33
*** ChuckC_ has joined #openstack-infra01:34
*** chuckC has joined #openstack-infra01:35
*** patrickeast has quit IRC01:38
*** Sukhdev has quit IRC01:38
*** sdake_ has quit IRC01:38
*** marcoemorais has quit IRC01:39
*** unicell has joined #openstack-infra01:42
*** sdake_ has joined #openstack-infra01:44
*** mikedillion has quit IRC01:47
openstackgerritMatthew Treinish proposed a change to openstack-infra/subunit2sql: Add migration tests with data  https://review.openstack.org/12493801:50
openstackgerritMatthew Treinish proposed a change to openstack-infra/subunit2sql: Preseed data for migration tests until migration 163fd5aa1380  https://review.openstack.org/12493901:50
*** baoli has quit IRC01:50
*** tsg has joined #openstack-infra01:51
openstackgerritMatthew Treinish proposed a change to openstack-infra/subunit2sql: Add order column to run table  https://review.openstack.org/11786101:51
openstackgerritMatthew Treinish proposed a change to openstack-infra/subunit2sql: Add db api function to get the latest run  https://review.openstack.org/11788201:51
openstackgerritMatthew Treinish proposed a change to openstack-infra/subunit2sql: Add session objects for get_all* db api calls  https://review.openstack.org/11841901:52
openstackgerritMatthew Treinish proposed a change to openstack-infra/subunit2sql: Add migration tests with data  https://review.openstack.org/12493801:53
openstackgerritMatthew Treinish proposed a change to openstack-infra/subunit2sql: Preseed data for migration tests until migration 163fd5aa1380  https://review.openstack.org/12493901:53
openstackgerritMatthew Treinish proposed a change to openstack-infra/subunit2sql: Add session objects for get_all* db api calls  https://review.openstack.org/11841901:53
openstackgerritMatthew Treinish proposed a change to openstack-infra/subunit2sql: Add order column to run table  https://review.openstack.org/11786101:53
openstackgerritMatthew Treinish proposed a change to openstack-infra/subunit2sql: Add db api function to get the latest run  https://review.openstack.org/11788201:53
*** Ryan_Lane has quit IRC01:56
*** nosnos has joined #openstack-infra01:56
*** signed8bit has joined #openstack-infra01:56
*** unicell has quit IRC01:59
*** _buzz has joined #openstack-infra02:04
*** ivar-laz_ has joined #openstack-infra02:05
*** bdpayne has joined #openstack-infra02:07
*** daya_k has joined #openstack-infra02:08
*** ivar-lazzaro has quit IRC02:09
*** aysyanne has quit IRC02:09
*** ivar-laz_ has quit IRC02:09
*** markmcclain has quit IRC02:10
*** signed8bit has quit IRC02:11
*** shashankhegde has joined #openstack-infra02:22
*** unicell has joined #openstack-infra02:25
*** dimtruck is now known as zz_dimtruck02:25
*** arosen1 has joined #openstack-infra02:25
*** otter768 has joined #openstack-infra02:26
openstackgerritA change was merged to openstack-infra/subunit2sql: Stop using intersphinx  https://review.openstack.org/12397302:28
*** JayJ has quit IRC02:30
*** arosen1 has left #openstack-infra02:33
*** daya_k has quit IRC02:35
*** melwitt1 has quit IRC02:37
*** dims has quit IRC02:42
*** dims has joined #openstack-infra02:43
*** tsg has quit IRC02:43
pleia2AJaeger: thanks :)02:44
*** mmaglana has joined #openstack-infra02:45
*** dims has quit IRC02:47
*** r1chardj0n3s is now known as r1chardj0n3s_afk02:47
*** sdake_ has quit IRC02:47
*** kgavini has quit IRC02:49
*** shashankhegde has quit IRC02:54
*** pcrews has quit IRC02:56
*** pcrews has joined #openstack-infra02:57
*** dims has joined #openstack-infra02:57
*** daya_k has joined #openstack-infra02:58
jheskethsdague: let me know when you're around to discuss the zuul status page stuff03:01
openstackgerritMatthew Treinish proposed a change to openstack-infra/subunit2sql: Add migration tests with data  https://review.openstack.org/12493803:01
openstackgerritMatthew Treinish proposed a change to openstack-infra/subunit2sql: Preseed data for migration tests until migration 163fd5aa1380  https://review.openstack.org/12493903:02
openstackgerritMatthew Treinish proposed a change to openstack-infra/subunit2sql: Add session objects for get_all* db api calls  https://review.openstack.org/11841903:02
openstackgerritMatthew Treinish proposed a change to openstack-infra/subunit2sql: Add order column to run table  https://review.openstack.org/11786103:02
openstackgerritMatthew Treinish proposed a change to openstack-infra/subunit2sql: Add db api function to get the latest run  https://review.openstack.org/11788203:02
*** pcrews has quit IRC03:04
*** andreaf has quit IRC03:04
*** andreaf has joined #openstack-infra03:05
*** dims has quit IRC03:07
*** dims has joined #openstack-infra03:08
*** mestery has joined #openstack-infra03:08
*** koolhead17 has joined #openstack-infra03:10
*** dims has quit IRC03:12
*** bdpayne has quit IRC03:12
openstackgerritMatthew Treinish proposed a change to openstack-infra/subunit2sql: Add order column to run table  https://review.openstack.org/11786103:16
*** pcrews has joined #openstack-infra03:21
*** nosnos has quit IRC03:22
*** nosnos has joined #openstack-infra03:23
*** armax has joined #openstack-infra03:24
*** harlowja is now known as harlowja_away03:26
*** hdd has joined #openstack-infra03:26
*** _buzz has quit IRC03:27
*** nosnos has quit IRC03:27
*** camunoz has quit IRC03:31
*** harlowja_away is now known as harlowja03:32
*** mestery has quit IRC03:33
*** andreaf has quit IRC03:33
*** andreaf has joined #openstack-infra03:34
*** armax has quit IRC03:38
*** adalbas has quit IRC03:38
*** mestery has joined #openstack-infra03:40
openstackgerritElizabeth K. Joseph proposed a change to openstack-infra/project-config: Move python33 test for Keystone to experimental  https://review.openstack.org/12456303:42
*** r1chardj0n3s_afk is now known as r1chardj0n3s03:42
*** zns has quit IRC03:42
morganfainbergpleia2, thanks for the spell-check :)03:43
*** zns has joined #openstack-infra03:43
pleia2sure thing :)03:43
*** mestery has quit IRC03:47
*** zns has quit IRC03:48
*** otter768 has quit IRC03:48
*** Ryan_Lane has joined #openstack-infra03:54
*** Ryan_Lane has quit IRC03:58
*** pcrews has quit IRC03:59
*** rushiagr_away is now known as rushiagr04:01
*** mrmartin has joined #openstack-infra04:02
*** marcoemorais has joined #openstack-infra04:03
*** marcoemorais1 has joined #openstack-infra04:05
*** KanagarajM has joined #openstack-infra04:07
*** marcoemorais has quit IRC04:08
*** otherwiseguy has quit IRC04:08
*** garyh has quit IRC04:10
openstackgerritMatthew Treinish proposed a change to openstack-infra/subunit2sql: Add migration tests with data  https://review.openstack.org/12493804:10
openstackgerritMatthew Treinish proposed a change to openstack-infra/subunit2sql: Preseed data for migration tests until migration 163fd5aa1380  https://review.openstack.org/12493904:10
openstackgerritMatthew Treinish proposed a change to openstack-infra/subunit2sql: Add session objects for get_all* db api calls  https://review.openstack.org/11841904:10
openstackgerritMatthew Treinish proposed a change to openstack-infra/subunit2sql: Add order column to run table  https://review.openstack.org/11786104:10
openstackgerritMatthew Treinish proposed a change to openstack-infra/subunit2sql: Add db api function to get the latest run  https://review.openstack.org/11788204:10
*** marcoemorais1 has left #openstack-infra04:11
*** nosnos has joined #openstack-infra04:14
*** otherwiseguy has joined #openstack-infra04:18
*** shashankhegde has joined #openstack-infra04:20
*** HeOS has joined #openstack-infra04:22
*** ildikov_ has quit IRC04:28
*** camunoz has joined #openstack-infra04:29
*** otherwiseguy has quit IRC04:29
*** yfried has quit IRC04:30
*** KanagarajM has quit IRC04:35
*** Daisy has joined #openstack-infra04:35
*** unicell has quit IRC04:37
*** yamahata has quit IRC04:40
*** yamahata has joined #openstack-infra04:41
*** mestery has joined #openstack-infra04:41
*** mrmartin has quit IRC04:48
*** mrmartin has joined #openstack-infra04:48
openstackgerritMatthew Treinish proposed a change to openstack-infra/subunit2sql: Add order column to run table  https://review.openstack.org/11786104:48
*** mrmartin has quit IRC04:48
openstackgerritMatthew Treinish proposed a change to openstack-infra/subunit2sql: Add db api function to get the latest run  https://review.openstack.org/11788204:49
*** rushiagr is now known as rushiagr_away04:51
*** gokrokve has joined #openstack-infra04:56
*** patrickeast has joined #openstack-infra04:57
*** mestery has quit IRC04:57
*** unicell has joined #openstack-infra04:58
*** mestery has joined #openstack-infra04:59
*** YorikSar has quit IRC05:01
*** loki184 has joined #openstack-infra05:01
*** hdd has quit IRC05:02
*** YorikSar has joined #openstack-infra05:03
*** KanagarajM has joined #openstack-infra05:03
*** patrickeast has quit IRC05:04
*** patrickeast has joined #openstack-infra05:05
*** patrickeast has quit IRC05:09
*** mmedvede has quit IRC05:12
*** patrickeast has joined #openstack-infra05:13
*** reed has quit IRC05:16
*** ildikov has joined #openstack-infra05:19
*** KanagarajM2 has joined #openstack-infra05:19
*** KanagarajM has quit IRC05:20
*** andreaf has quit IRC05:20
*** andreaf has joined #openstack-infra05:21
*** HeOS has quit IRC05:21
*** unicell has quit IRC05:24
*** yfried has joined #openstack-infra05:28
*** sressot has joined #openstack-infra05:30
*** rushiagr_away is now known as rushiagr05:32
*** dalgaaf__ has joined #openstack-infra05:35
*** timrc is now known as timrc-afk05:35
*** mestery has quit IRC05:36
*** waynr has quit IRC05:39
*** patrickeast has quit IRC05:41
*** cristov_mac has quit IRC05:42
*** linuxmohan has joined #openstack-infra05:44
*** linuxmohan has quit IRC05:44
*** linuxmohan has joined #openstack-infra05:45
*** andreaf has quit IRC05:46
*** berendt has joined #openstack-infra05:47
*** praneshp has quit IRC05:53
*** ivar-lazzaro has joined #openstack-infra06:02
*** ivar-lazzaro has joined #openstack-infra06:02
*** harlowja is now known as harlowja_away06:05
*** waynr has joined #openstack-infra06:06
*** vigneshvar has joined #openstack-infra06:08
*** unicell has joined #openstack-infra06:08
*** rwsu has quit IRC06:09
*** SumitNaiksatam_ has joined #openstack-infra06:11
*** Ryan_Lane has joined #openstack-infra06:11
*** SumitNaiksatam has quit IRC06:14
*** SumitNaiksatam_ is now known as SumitNaiksatam06:14
*** SumitNaiksatam has quit IRC06:21
*** SumitNaiksatam has joined #openstack-infra06:22
*** shashankhegde has quit IRC06:22
*** stevemar has quit IRC06:24
*** emagana has joined #openstack-infra06:26
AJaegerflaper87|afk, once you're back online, could you review https://review.openstack.org/#/c/116314/ again, please?06:26
*** praneshp has joined #openstack-infra06:27
*** k4n0 has joined #openstack-infra06:27
*** SumitNaiksatam has quit IRC06:29
*** praneshp_ has joined #openstack-infra06:30
*** ashaeron has joined #openstack-infra06:30
*** SumitNaiksatam has joined #openstack-infra06:30
*** praneshp has quit IRC06:32
*** praneshp_ is now known as praneshp06:32
*** emagana has quit IRC06:33
*** emagana has joined #openstack-infra06:34
*** SumitNaiksatam has quit IRC06:34
*** SumitNaiksatam has joined #openstack-infra06:35
*** emagana has quit IRC06:39
*** nosnos has quit IRC06:39
*** nosnos has joined #openstack-infra06:40
*** nosnos has quit IRC06:41
*** atuvenie has joined #openstack-infra06:41
*** nosnos_ has joined #openstack-infra06:41
*** Longgeek has joined #openstack-infra06:43
*** Longgeek has quit IRC06:44
*** wenlock has quit IRC06:44
*** Longgeek has joined #openstack-infra06:46
openstackgerritAdam Gandelman proposed a change to openstack-infra/elastic-recheck: Add query for 1375488  https://review.openstack.org/12491306:46
*** jcoufal has joined #openstack-infra06:47
*** jcoufal has quit IRC06:47
*** Longgeek_ has joined #openstack-infra06:49
*** camunoz has quit IRC06:51
*** GheRivero has quit IRC06:52
*** Longgeek has quit IRC06:52
*** mmaglana has quit IRC06:55
openstackgerritYair Fried proposed a change to openstack-infra/devstack-gate: Adds tenant_network_conf to tempest.conf  https://review.openstack.org/12476806:58
*** GheRivero has joined #openstack-infra06:58
*** xchu has quit IRC07:03
*** hashar has joined #openstack-infra07:06
*** arxcruz has joined #openstack-infra07:12
*** arxcruz has quit IRC07:14
*** mmagr has joined #openstack-infra07:16
*** arxcruz has joined #openstack-infra07:17
*** koolhead17 has quit IRC07:17
*** _nadya_ has joined #openstack-infra07:18
*** VijayTripathi has quit IRC07:18
*** jcoufal has joined #openstack-infra07:22
*** HeOS has joined #openstack-infra07:22
*** _nadya_ has joined #openstack-infra07:23
*** jcoufal has quit IRC07:23
*** e0ne has joined #openstack-infra07:23
*** jcoufal has joined #openstack-infra07:23
*** e0ne has quit IRC07:24
*** gokrokve has quit IRC07:26
*** rdopieralski has joined #openstack-infra07:26
*** Longgeek has joined #openstack-infra07:27
*** ivar-lazzaro has quit IRC07:27
*** Longgeek_ has quit IRC07:28
*** _nadya_ has quit IRC07:28
*** e0ne has joined #openstack-infra07:29
*** yamamoto has quit IRC07:29
*** Guest68510 has joined #openstack-infra07:30
*** r1chardj0n3s is now known as r1chardj0n3s_afk07:31
openstackgerritMate Lakat proposed a change to openstack-infra/project-config: XenServer: Add XenServer support  https://review.openstack.org/12473007:32
*** jcoufal has quit IRC07:32
*** ZZelle has quit IRC07:34
*** jcoufal has joined #openstack-infra07:34
*** ZZelle has joined #openstack-infra07:34
*** koolhead17 has joined #openstack-infra07:35
*** koolhead17 has quit IRC07:35
*** koolhead17 has joined #openstack-infra07:35
openstackgerritA change was merged to openstack-infra/infra-manual: Add an introduction paragraph to the developer guide  https://review.openstack.org/10702407:36
*** jp_at_hp has joined #openstack-infra07:36
*** bogdando has quit IRC07:39
*** jcoufal has quit IRC07:40
*** sunrenjie6 has joined #openstack-infra07:40
*** jcoufal has joined #openstack-infra07:40
*** Longgeek has quit IRC07:40
*** Longgeek has joined #openstack-infra07:42
openstackgerritThierry Carrez proposed a change to openstack-infra/release-tools: Fix rcdelivery.sh using wrong final version  https://review.openstack.org/12496307:42
*** ZZelle has quit IRC07:42
*** ZZelle has joined #openstack-infra07:43
openstackgerritChristoffer Skeppstedt proposed a change to openstack-infra/jenkins-job-builder: Serialize git-revision param for trigger_builds  https://review.openstack.org/12496507:46
*** bogdando has joined #openstack-infra07:46
openstackgerritJoshua Hesketh proposed a change to openstack-infra/project-config: Run gate-{name}-docs on infra-publish-jobs  https://review.openstack.org/12496607:46
*** e0ne has quit IRC07:47
*** skeppstedt has joined #openstack-infra07:50
*** mpaolino has joined #openstack-infra07:51
*** jistr has joined #openstack-infra07:55
openstackgerritJordan Pittier proposed a change to openstack-infra/project-config: Add pep8 gate job to stackforge/ospurge  https://review.openstack.org/12472907:57
*** andreykurilin_ has joined #openstack-infra07:57
*** jlibosva has joined #openstack-infra07:58
*** gokrokve has joined #openstack-infra07:58
*** zz_johnthetubagu has joined #openstack-infra07:59
*** zz_johnthetubagu is now known as johnthetubaguy07:59
*** yamamoto has joined #openstack-infra07:59
*** gokrokve has quit IRC08:00
*** yamamoto has quit IRC08:00
*** gokrokve has joined #openstack-infra08:00
*** loki184 has quit IRC08:02
*** e0ne has joined #openstack-infra08:02
*** YorikSar has quit IRC08:02
*** derekh has joined #openstack-infra08:03
*** boden has joined #openstack-infra08:03
*** YorikSar has joined #openstack-infra08:04
*** gokrokve has quit IRC08:05
*** ihrachyshka has joined #openstack-infra08:07
*** e0ne has quit IRC08:13
*** ociuhandu has quit IRC08:17
*** e0ne has joined #openstack-infra08:17
*** e0ne has quit IRC08:18
*** afazekas has joined #openstack-infra08:18
*** jgallard has joined #openstack-infra08:19
*** MaxV has joined #openstack-infra08:19
*** loki184 has joined #openstack-infra08:23
*** cdent has joined #openstack-infra08:30
*** amuller has joined #openstack-infra08:34
*** doude has joined #openstack-infra08:39
*** dizquierdo has joined #openstack-infra08:41
*** andreykurilin_ has quit IRC08:44
openstackgerritMate Lakat proposed a change to openstack-infra/project-config: XenServer: Add XenServer support  https://review.openstack.org/12473008:50
*** andreaf_ is now known as andreaf08:59
*** yfried is now known as yfried|afk08:59
*** gokrokve has joined #openstack-infra08:59
*** sdake_ has joined #openstack-infra09:00
*** gokrokve has quit IRC09:04
*** sdake_ has quit IRC09:05
*** e0ne has joined #openstack-infra09:14
*** ociuhandu has joined #openstack-infra09:14
*** rlandy has joined #openstack-infra09:15
*** mancdaz has joined #openstack-infra09:17
*** che-arne has joined #openstack-infra09:24
*** yfried|afk is now known as yfried09:26
*** e0ne has quit IRC09:30
*** alexpilotti has joined #openstack-infra09:33
*** Daisy has quit IRC09:34
*** praneshp has quit IRC09:34
*** liusheng has quit IRC09:40
*** e0ne has joined #openstack-infra09:43
*** salv-orlando has joined #openstack-infra09:49
openstackgerritSergey Skripnick proposed a change to openstack-infra/project-config: Add rally-fakevirt performance job  https://review.openstack.org/12499609:52
sdaguejhesketh: up now09:55
*** koolhead17 has quit IRC09:58
*** gokrokve has joined #openstack-infra09:59
sbadia\o hi infra10:01
sbadiacould I have a review on this two patchs ? ( https://review.openstack.org/#/c/124569/ for metadata.json on puppet-lint jobs, and https://review.openstack.org/#/c/124570/ for default jenkins builders for puppet-vswitch ) ?10:01
sbadiathanks in advance =)10:02
openstackgerritRicardo Carrillo Cruz proposed a change to openstack-infra/project-config: Add version-properties builder to macros.yaml  https://review.openstack.org/12436510:02
*** yfried_ has joined #openstack-infra10:02
*** gokrokve has quit IRC10:03
*** yfried has quit IRC10:03
*** _nadya_ has joined #openstack-infra10:08
*** yfried_ is now known as yfried|afk10:18
*** hashar has quit IRC10:19
*** sandywalsh has quit IRC10:20
*** yfried|afk is now known as yfried_10:22
*** sandywalsh has joined #openstack-infra10:22
*** jcoufal has quit IRC10:23
*** jcoufal has joined #openstack-infra10:24
*** pcm_ has joined #openstack-infra10:30
*** pcm_ has quit IRC10:31
*** pcm_ has joined #openstack-infra10:32
*** yfried_ has quit IRC10:34
*** pelix has joined #openstack-infra10:34
openstackgerritSean Dague proposed a change to openstack-infra/zuul: add support for getting status of individual changes  https://review.openstack.org/12439010:34
openstackgerritSean Dague proposed a change to openstack-infra/zuul: add support for filtering changes in zuul webapp  https://review.openstack.org/12500210:34
*** yamahata has quit IRC10:35
*** yfried has joined #openstack-infra10:40
*** MindfulMonk has quit IRC10:46
*** jgallard has quit IRC10:52
*** MindfulMonk has joined #openstack-infra10:53
*** habib__ has joined #openstack-infra10:54
*** ildikov_ has joined #openstack-infra10:57
*** MaxV has quit IRC10:57
*** MaxV has joined #openstack-infra10:58
*** gokrokve has joined #openstack-infra10:59
*** ildikov has quit IRC11:01
*** MaxV has quit IRC11:02
*** loki184 has quit IRC11:02
*** gokrokve has quit IRC11:04
*** yfried_ has joined #openstack-infra11:04
*** yfried__ has joined #openstack-infra11:05
*** MaxV has joined #openstack-infra11:05
*** yfried has quit IRC11:05
*** hashar has joined #openstack-infra11:08
*** dims has joined #openstack-infra11:09
*** yfried_ has quit IRC11:09
*** ildikov_ has quit IRC11:12
*** ildikov has joined #openstack-infra11:12
*** e0ne has quit IRC11:13
*** signed8bit has joined #openstack-infra11:21
*** zz_dimtruck is now known as dimtruck11:22
*** mikal has quit IRC11:23
*** loquacities has quit IRC11:23
*** yfried__ is now known as yfried|afk11:28
*** signed8bit has quit IRC11:29
*** jistr is now known as jistr|english11:31
openstackgerritRicardo Carrillo Cruz proposed a change to openstack-infra/project-config: The current maven jobs in maven-plugin-jobs work fine for simple projects, but it cannot work for multi-module projects where multiple artifacts are generated with their corresponding pom.xml files. Also, it is not possible to pass project-specific settin  https://review.openstack.org/12501111:33
*** mrda-vacation has quit IRC11:33
*** MaxV has quit IRC11:34
*** MaxV has joined #openstack-infra11:34
openstackgerritRicardo Carrillo Cruz proposed a change to openstack-infra/project-config: Add support for multi-module maven jobs  https://review.openstack.org/12501111:34
*** mikal has joined #openstack-infra11:35
jheskethsdague: left some comments on your patch.. I was going to help re-roll but wanted to check on your thoughts re the json structure (my main concern)11:36
sdaguejhesketh: so I respun11:36
sdaguebased on your comments11:37
jheskethyep, just loading11:37
sdaguemostly I'd like to not get hung up on filtering structure to land the individual status patch11:37
*** timrc-afk is now known as timrc11:37
sdaguebecause the individual status patch, once in, means I can write it into gerrit ui later the same day11:38
jheskethsdague: okay, sounds good to me11:40
sdagueI think that makes patch 1 landable, and patch 2 something to discuss11:40
jheskethsure11:40
sdaguebecause in the gate queue case, what would you expect to return for project=openstack/nova ?11:40
*** sressot has quit IRC11:40
*** sandywalsh_ has joined #openstack-infra11:41
*** yolanda has quit IRC11:41
*** _nadya_ has quit IRC11:42
jheskethsdague: what are your thoughts on having existing webapp tests into the new TestWebApp?11:42
sdaguejhesketh: I'd be happy to migrate them over11:42
sdagueI think it's a follow on patch though11:42
jheskethyeah, I'm not worried about it11:42
jheskethsdague: I was expecting it to return a json blob similar to how the javascript app filters things11:43
*** sandywalsh has quit IRC11:43
*** sressot has joined #openstack-infra11:43
sdaguejhesketh: ok, except that means that if you filter gate by openstack/nova11:43
*** yolanda has joined #openstack-infra11:43
sdagueyou get the whole gate queue11:43
sdaguewhich... isn't much filtering11:43
jheskethoh, I see your point11:43
jheskethsdague: well we wouldn't necessarily have to include the heads items (which is what I Thought the web app did, but testing it it appears to return the whole queue)11:44
*** kgiusti has joined #openstack-infra11:45
sdagueyeh11:45
*** dhellmann has quit IRC11:48
*** yfried|afk is now known as yfried__11:48
*** jhesketh_ has joined #openstack-infra11:49
*** dhellmann has joined #openstack-infra11:50
*** isviridov is now known as isviridov_away11:50
*** jhesketh has quit IRC11:51
ZZelleadam_g, hi11:51
*** weshay has joined #openstack-infra11:53
*** tsg has joined #openstack-infra11:54
*** mbacchi has joined #openstack-infra11:57
*** jhesketh has joined #openstack-infra11:58
*** skeppste_ has joined #openstack-infra11:58
*** yfried__ is now known as yfried|afk11:58
openstackgerritJulien Danjou proposed a change to openstack-dev/hacking: Add a check for old style class declarations  https://review.openstack.org/11684611:58
*** skeppste_ has quit IRC11:58
*** skeppste_ has joined #openstack-infra11:59
*** jhesketh_ has quit IRC11:59
*** baoli has joined #openstack-infra11:59
*** gokrokve has joined #openstack-infra11:59
*** jhesketh has quit IRC12:00
*** skeppste_ has quit IRC12:00
*** skeppst__ has joined #openstack-infra12:00
*** gokrokve has quit IRC12:00
*** jhesketh has joined #openstack-infra12:01
*** gokrokve has joined #openstack-infra12:01
*** skeppstedt has quit IRC12:01
jheskethsdague: so I'm unsure on the filter thing.. maybe we need another opinion there?12:01
*** gus has quit IRC12:06
*** gokrokve has quit IRC12:06
*** tsg has quit IRC12:06
*** yfried|afk has quit IRC12:06
sdaguejhesketh: yep, I'm fine with holding that for now12:08
sdagueas long as we move forward on the single status case12:09
sdaguemaybe we need to know what the use case is12:09
*** KanagarajM2 has quit IRC12:10
jheskethsounds good to me12:11
jheskethsdague: looks like we were both working towards the same thing anyway... I actually had an overlay of zuul results onto gerrit from over 6 months back that kept getting pushed back for various reasons12:11
sdagueyeh12:12
sdagueso I think we can work this into the hideci.js code12:12
sdaguebecause ideally it would live above that12:12
sdaguehonestly... I wouldn't replicate the whole test block from zuul12:12
sdaguejust a small table to know that changes are currently running12:13
*** yfried|afk has joined #openstack-infra12:17
*** dprince has joined #openstack-infra12:18
*** dkranz has joined #openstack-infra12:19
*** mmedvede has joined #openstack-infra12:19
*** mattoliverau has quit IRC12:19
*** zns has joined #openstack-infra12:19
*** e0ne has joined #openstack-infra12:19
*** zns has quit IRC12:22
*** yamahata has joined #openstack-infra12:23
*** bswartz has quit IRC12:24
*** aysyanne has joined #openstack-infra12:25
*** dizquierdo has quit IRC12:26
*** ihre has joined #openstack-infra12:27
*** dimtruck is now known as zz_dimtruck12:28
*** dims has quit IRC12:29
*** dims has joined #openstack-infra12:30
*** yaguang has quit IRC12:30
*** yfried|afk has quit IRC12:31
*** waterkinfe has joined #openstack-infra12:33
*** KanagarajM has joined #openstack-infra12:34
jheskethsdague: yep, agreed12:34
*** waterkinfe has quit IRC12:34
*** unicell has quit IRC12:35
*** waterkinfe has joined #openstack-infra12:35
*** adalbas has joined #openstack-infra12:35
*** skeppst__ has quit IRC12:35
sdagueaddressing your comments in hideci.js now12:36
sdagueI have timing numbers, ci_parse_results is not nearly as expensive as people imagine :)12:36
openstackgerritSean Dague proposed a change to openstack-infra/config: refactor of hideci for readability and modularity  https://review.openstack.org/12463212:41
sdagueso on a short review page it's 5ms12:41
*** dizquierdo has joined #openstack-infra12:41
*** tsg has joined #openstack-infra12:41
sdagueon this review - https://review.openstack.org/#/c/40467/12:42
sdagueit was 85ms12:42
*** rdopieralski has left #openstack-infra12:42
sdaguethat has 103 change revisions12:42
gilliardsdague: about the code being called twice - it may be browser dependent, but it definitely happens to me with firefox.12:42
sdaguewhich is the largest I've ever seen12:42
sdaguegilliard: hmmmm12:42
sdagueso how did it ever work before?12:42
sdaguebecause the toggle would have gotten called twice12:43
sdagueso everything would be displayed again12:43
gilliard1st time, the style attribute changes, 2nd time one of the aria-* attributes has changed.  put a console.log in ci_page_loaded to see.12:44
sdagueso... why do we have the mutation observer anyway?12:48
openstackgerritA change was merged to openstack-infra/jenkins-job-builder: Add PMD publisher  https://review.openstack.org/11831212:48
*** mjturek has joined #openstack-infra12:48
*** zz_dimtruck is now known as dimtruck12:48
gilliardIt fires (a lot) when anything on the page changes, I think.  In particular there, it is looking for the "waiting..." message to disappear.12:48
*** skeppstedt has joined #openstack-infra12:48
sdagueoh... is this because .... gerrit is one big js bundle12:49
*** miqui has joined #openstack-infra12:49
gilliardnothing was toggled on page load.  It just decided whether or not to show the button which does the toggle when clicked.12:49
sdaguegilliard: so, this is the way the code was previously12:49
gilliardagreed.12:50
gilliardJust pointing it out.  I was planning to put up a patch for it in the next few hours if you'd like.12:50
sdagueif you have a fix for it, go for it :)12:50
sdagueok12:50
gilliardcool.12:50
sdagueI think I addressed the rest of the comments12:50
sdagueit's harder to test live now, because comments are cached12:51
*** rushiagr is now known as rushiagr_away12:51
*** unicell has joined #openstack-infra12:51
sdagueactually, let me do one more function change, then it's testable better12:52
*** changbl has quit IRC12:52
sdaguegilliard: I wonder....12:53
sdagueis there reason you are getting double firing because you are redefining things later?12:53
gilliardI don't think so.12:54
*** _nadya_ has joined #openstack-infra12:54
gilliardI'll paste.o.o something...12:54
openstackgerritChristian Berendt proposed a change to openstack-infra/devstack-gate: Use $((EXPRESSION)) instead of $[EXPRESSION]  https://review.openstack.org/12503812:55
*** JayJ has joined #openstack-infra12:55
*** jistr|english is now known as jistr12:56
gilliardhttp://paste.openstack.org/show/117111/12:57
gilliardstyle changes then aria-hidden changes.12:57
*** koolhead17 has joined #openstack-infra12:58
gilliardI don't know why we have to use this mutationObserver anyway, I assume that the JS GWT puts out is hostile enough to extension so we have to watch for side effects.  I'll investigate & put up another patch later.12:58
*** ildikov has quit IRC12:59
*** gokrokve has joined #openstack-infra12:59
*** bswartz has joined #openstack-infra12:59
*** ildikov has joined #openstack-infra12:59
*** k4n0 has quit IRC13:01
*** gokrokve has quit IRC13:03
*** ildikov has quit IRC13:04
*** cnesa15 has joined #openstack-infra13:04
*** ildikov has joined #openstack-infra13:05
*** isviridov_away is now known as isviridov13:06
*** homeless has joined #openstack-infra13:06
*** krtaylor has quit IRC13:06
*** ppai has joined #openstack-infra13:06
openstackgerritSean Dague proposed a change to openstack-infra/config: refactor of hideci for readability and modularity  https://review.openstack.org/12463213:07
sdagueok, so ... ^^^ should just fix it13:07
sdagueby not using toggle on page load13:07
*** krtaylor has joined #openstack-infra13:07
sdaguebut actually forcing the hide13:07
sdaguerepurposing toggle was the wrong call anyway13:07
* gilliard looking13:08
openstackgerritSean Dague proposed a change to openstack-infra/config: refactor of hideci for readability and modularity  https://review.openstack.org/12463213:08
sdagueoops, typo in function name13:08
sdagueI used the old one that wasn't there13:08
gilliardI'm trying to think of a way to test that the caching of "comments" var works as intended.13:08
sdagueyeh... me too13:09
sdagueit should13:09
*** julim has joined #openstack-infra13:09
sdagueit's closures13:09
sdaguethat being said, the caching is possibly completely overkill13:09
sdagueon the average gerrit review we're talking about 20ms13:10
sdaguethe paint calls way outstrip that13:10
gilliardWhat I want is an http proxy where I can inject this file, so that the window.onload stuff works as if it were live.13:10
*** denis_makogon has joined #openstack-infra13:11
*** changbl has joined #openstack-infra13:11
*** _buzz has joined #openstack-infra13:11
sdagueyeh13:11
sdagueif you figure out how to do that, we should write up that testing procedure in the doc13:12
sdaguein the repo somewhere13:12
sdaguethat being said, at a certain point, we probably just have to decide it's probably right, land it, then fix it if it isn't13:13
openstackgerrityolanda.robla proposed a change to openstack-infra/storyboard: Disable login for users  https://review.openstack.org/11217413:14
openstackgerritAlexander Kislitsky proposed a change to openstack-infra/project-config: Fuel-stats repo for Fuel  https://review.openstack.org/12477313:15
*** salv-orlando has quit IRC13:15
*** changbl has quit IRC13:18
*** dustins has joined #openstack-infra13:18
*** esker has joined #openstack-infra13:19
*** esker has quit IRC13:19
*** esker has joined #openstack-infra13:20
*** zns has joined #openstack-infra13:20
*** zns has quit IRC13:21
*** mriedem has joined #openstack-infra13:21
*** zns has joined #openstack-infra13:22
*** rushiagr_away is now known as rushiagr13:24
*** juzuluag has joined #openstack-infra13:24
*** eglynn-officeafk is now known as eglynn-office13:25
openstackgerrityolanda.robla proposed a change to openstack-infra/storyboard: Disable login for users  https://review.openstack.org/11217413:26
*** dkliban_afk is now known as dkliban13:29
*** zz_jgrimm is now known as jgrimm13:29
*** vdo has joined #openstack-infra13:29
openstackgerrityolanda.robla proposed a change to openstack-infra/storyboard: Disable login for users  https://review.openstack.org/11217413:31
*** doug-fish has joined #openstack-infra13:33
*** KanagarajM has quit IRC13:33
*** rushiagr is now known as rushiagr_away13:36
*** yfried|afk has joined #openstack-infra13:37
*** mfer has joined #openstack-infra13:39
openstackgerritDenis M. proposed a change to openstack-infra/project-config: Added jobs for MongoDB and Percona datastores  https://review.openstack.org/12505713:40
*** skeppstedt has quit IRC13:40
*** skeppstedt has joined #openstack-infra13:41
*** jgallard has joined #openstack-infra13:43
*** sdake has quit IRC13:44
*** sdake has joined #openstack-infra13:44
*** skeppstedt has quit IRC13:45
openstackgerritA change was merged to openstack-infra/jenkins-job-builder: Adding support for the Powershell plugin.  https://review.openstack.org/12114213:45
*** dimtruck is now known as zz_dimtruck13:47
*** radez_g0n3 is now known as radez13:49
*** otherwiseguy has joined #openstack-infra13:51
*** waterkinfe has quit IRC13:51
*** skeppstedt has joined #openstack-infra13:52
*** sdake_ has joined #openstack-infra13:53
*** vigneshvar has quit IRC13:53
*** dangers_away is now known as dangers13:54
*** markmcclain has joined #openstack-infra13:54
*** markmcclain has quit IRC13:54
*** dkranz has quit IRC13:55
*** nosnos_ has quit IRC13:59
*** nosnos has joined #openstack-infra13:59
*** gokrokve has joined #openstack-infra13:59
*** zaphod42 has quit IRC13:59
*** dustins has quit IRC14:00
*** wenlock has joined #openstack-infra14:00
*** sdake_ has quit IRC14:01
*** dustins has joined #openstack-infra14:02
*** dane_leblanc has joined #openstack-infra14:02
*** zz_gondoi is now known as gondoi14:02
*** bswartz has quit IRC14:03
*** linuxmohan has quit IRC14:03
*** nosnos has quit IRC14:04
*** kashyap has quit IRC14:04
*** gokrokve has quit IRC14:04
*** Sincler has joined #openstack-infra14:05
*** e0ne has quit IRC14:06
*** daya_k has quit IRC14:06
*** jergerber has joined #openstack-infra14:09
*** andreaf_ has joined #openstack-infra14:10
openstackgerritA change was merged to openstack-infra/jenkins-job-builder: Flatten attribute for publish plugins  https://review.openstack.org/12408114:11
*** andreaf has quit IRC14:13
*** andreaf_ is now known as andreaf14:14
*** andreaf_ has joined #openstack-infra14:14
*** zz_dimtruck is now known as dimtruck14:15
*** tonytan4ever has joined #openstack-infra14:16
*** jergerber has quit IRC14:16
*** dmsimard_away is now known as dmsimard14:16
*** hdd has joined #openstack-infra14:17
*** chuckC has quit IRC14:18
*** BOKALDO has joined #openstack-infra14:18
*** rushiagr_away is now known as rushiagr14:19
*** pbelanyi_ is now known as pbelanyi14:21
openstackgerritA change was merged to openstack-infra/jenkins-job-builder: ModuleRegistry now cache entry points group  https://review.openstack.org/12316314:22
anteayaAJaeger: the job that adds more layout checks for zuul/layout.yaml: https://review.openstack.org/#/c/124593/1 this patch has the script for the checks and adds it to tox, what is your plan for having it run in check or gate?14:22
*** ildikov_ has joined #openstack-infra14:23
*** ildikov has quit IRC14:23
openstackgerritSean Dague proposed a change to openstack-infra/elastic-recheck: also ignore icehouse partial grenade  https://review.openstack.org/12507014:24
*** alexpilotti has quit IRC14:24
*** BOKALDO has quit IRC14:26
*** andreaf has quit IRC14:26
*** ChuckC_ has quit IRC14:27
*** gokrokve has joined #openstack-infra14:27
*** bradjones has quit IRC14:27
*** xyang0 has joined #openstack-infra14:28
*** cnesa15 has quit IRC14:29
*** otherwiseguy has quit IRC14:29
*** e0ne has joined #openstack-infra14:29
*** gokrokve has quit IRC14:30
*** gokrokve has joined #openstack-infra14:31
*** eharney has joined #openstack-infra14:31
*** jcoufal has quit IRC14:33
*** datsun180b has joined #openstack-infra14:33
*** mestery has joined #openstack-infra14:33
anteayaBobBall: when you get a chance can we have a bit of a chat about: https://review.openstack.org/#/c/124730/314:34
openstackgerritA change was merged to openstack-infra/elastic-recheck: also ignore icehouse partial grenade  https://review.openstack.org/12507014:34
BobBallsure anteaya14:34
anteayathese scripts appear to be specific to xenserver14:34
anteayathanks14:34
BobBallThey are14:34
anteayaand we recently cleared out some code from master that was specific to hp and rackspace14:34
anteayahelp me understand why these scripts beneifit everyone using openstack14:35
anteayaand should be in master14:35
*** emagana has joined #openstack-infra14:35
BobBallThey only are useful for XenServer - we can host them elsewhere if needed.14:35
mordredanteaya: I went on the hp/rackspace rampage because those are vendors who should just be compatible with openstack14:35
*** krtaylor has quit IRC14:36
mordredanteaya: for open source software drivers, I think we _do_ want to have support for them in our tree14:36
mordredanteaya: similar to mysql vs. postgres14:36
anteayamordred: any objection to bob's scripts?14:36
*** wenlock has quit IRC14:36
mordrednot at all14:37
anteayathank you14:37
anteayathat was what I needed14:37
anteayathanks BobBall14:37
BobBallok, great14:37
mordredI look forward to the day when we're as good at testing xenserver as we are with testing kvm14:37
*** krtaylor has joined #openstack-infra14:37
*** jcoufal has joined #openstack-infra14:37
mordredBobBall: btw - did you see that the dib-nodepool stuff landed?14:37
BobBallyes mordred14:37
mordredcool. I have no idea if that makes things harder, easier, or indifferent14:37
BobBallWe've got a task in Q4 to look at moving to dib (well, not actually using DIB but a pre-built image)14:37
mordred++14:38
BobBallIn fact, not using DIB at all, but extending nodepool to just be able to upload an image to glance with no DIB actions14:38
mordred(strangely, it'll still probably use dib on our side - but the dib element will be VERY small)14:38
mordredoh14:38
BobBall(which it might be able to do ATM - not looked sufficiently)14:38
mordredgotcha14:38
*** garyh has joined #openstack-infra14:38
mordredno, it has not - but your way is better than my way14:38
fungiis xenserver actually 100% free software? i've used xen a bunch in the past, but never xenserver14:38
BobBallThe XenServer image is a right pain at the best of times...14:38
BobBallyes fungi14:38
BobBall100% free and open source14:39
fungitheir "download for free" page requiring a login is a bit of a turn-off https://store.citrix.com/store/citrix/en_US/DisplayThreePgCheckoutAddressPaymentInfoPage/ThemeID.2816960014:39
anteayaBobBall: ah and that I didn't know14:39
anteayaso I learned something14:39
BobBallwww.xenserver.org has _exactly_ the same builds as citrix.com14:39
mordredBobBall: you're still probably going to need dib I think14:39
*** bknudson has joined #openstack-infra14:39
BobBalldon't sue that one fungi :)14:39
*** reed has joined #openstack-infra14:39
fungiheh. is that the trap for suckers with money to waste?14:39
BobBallCitrix want to make a difference between 'XenServer' and 'Citrix XenServer' (which is the thing you can buy support on)14:39
mordredBobBall: because we're using dib to do the key injection stuff and to do the pre-caching of all the stuff we pre-cache14:39
*** david-lyle has joined #openstack-infra14:40
BobBallI like that suggestion fungi ;)14:40
mordredBUT - I may be very wrong14:40
mordredI'm still caffinating14:40
mordredMySQL did the same thing back in the day14:40
fungiBobBall: aha. got it. i was indeed conflating "xenserver" with "citrix xenserver." thanks for the info!14:40
mordredmay still do it14:40
BobBallmordred: Understood.  I hope we don't need to do that - it'll probably rule out DIB due to the convoluted architecture we have...14:40
mordredMySQL Enterprise was ... well ... absolutely indistinguishable on a binary level from MySQL Community14:40
mordredBobBall: I betcha lifeless will be happy to take that on as a challenge to solve if it is a problem :)14:41
*** _nadya_ has quit IRC14:41
BobBallGotta love companies and the way they make money ;)14:41
*** andreaf has joined #openstack-infra14:42
BobBallIndeed - well, Mate will be looking at it in a few weeks14:42
fungibut anyway, so just to simplify for my limited cranial capacity... xenserver is essentially a custom linux distro tuned for running/managing xen hypervisors?14:42
*** pcrews has joined #openstack-infra14:42
BobBallBingo fungi14:42
*** jedimike has joined #openstack-infra14:42
*** krtaylor has quit IRC14:42
*** openstackgerrit has quit IRC14:42
sdaguemordred: oh, so dib is currently blocking stable/icehouse requirements fixes14:43
sdaguemordred: if you want to noodle on why - https://review.openstack.org/#/c/124648/14:43
*** bradjones has joined #openstack-infra14:44
*** bradjones has quit IRC14:44
*** bradjones has joined #openstack-infra14:44
mordredsdague: I'd love to noodle on why14:44
*** krtaylor has joined #openstack-infra14:44
sdaguezuul change to expose individual patch set status -  https://review.openstack.org/#/c/124390/ - with lots of tests :)14:44
mordredsdague: why in the wORLD would anything break on that change?14:45
sdaguemordred: because dib is in projects.txt14:45
sdagueand probably doesn't have a stable/icehouse branch14:46
*** Sincler has quit IRC14:46
mordredsdague: so this actually has very little to do with pytz14:46
sdagueyep14:46
sdagueit has to do with dib14:46
mordredsomewhere we have an asymmetric gate on this14:46
mordredSpamapS: ^^ you awake?14:46
*** rdopieralski has joined #openstack-infra14:47
*** otherwiseguy has joined #openstack-infra14:47
sdagueyeh, honestly, my first instinct is just remove dib from projects.txt14:47
*** krtaylor has quit IRC14:47
*** markmcclain has joined #openstack-infra14:48
mordredsdague: so, I'm guessing that what happened is that the change to lande dib-utils in master happened and worked all fine14:48
sdagueyep14:48
mordredsdague: and that there have been no stable/icehouse changes since then14:48
rdopieralskihello, can I ask for adding the triple-o-core and horizon-core groups to the stackforge/tuskar-ui-extras-core group? https://review.openstack.org/#/admin/projects/stackforge/tuskar-ui-extras,access14:48
mordredyou know - honestly, this isn't just a dib thing, it's anything that doesn't have a stable branch, like the clients14:48
sdagueyep14:49
sdaguethe clients have a job for this though14:49
rdopieralskiso that we can actually use that repository :)14:49
mordredwe should probably run a stable branch job on master changes to requirements or something14:49
mordredsdague: do they?14:49
sdagueat least for icehouse14:49
sdagueI couldn't get the havana one to work, too much regression14:49
sdaguebut....14:49
sdagueI actually think that's the wrong thing14:49
mordredwhyfore?14:49
sdagueI think we should just remove things like dib from projects.txt14:50
mordredand the clients as well?14:50
sdagueI haven't decided on the clients yet14:50
mordredI mean, it's the same problem14:50
mordredeither we can solve it and it applies across the board, or we can't14:50
mordredI see no difference between dib and python-novaclient14:50
*** mmagr has quit IRC14:51
mordredfrom a build system perspective14:51
*** thedodd has joined #openstack-infra14:51
*** wenlock has joined #openstack-infra14:51
sdagueoh hey, power outage14:51
mordredwhee14:52
sdaguerebooted now, but I probably missed the last 2 minutes of conversation14:52
mordredsdague: since you left, I said:14:52
mordred14:50:32          mordred | I mean, it's the same problem14:52
mordred14:50:42          mordred | either we can solve it and it applies across the board, or we can't14:52
mordred14:50:53          mordred | I see no difference between dib and python-novaclient14:52
mordred14:51:03          mordred | from a build system perspective14:52
sdagueso... this opens the giant meta debate14:53
sdaguewhich projects have write authority for projects.txt14:53
sdagueis it everything?14:53
sdagueor is it the integrated release14:53
*** Sincler has joined #openstack-infra14:53
*** skeppstedt has quit IRC14:53
sdagueand if it's everything, why are we doing this :)14:54
*** skeppstedt has joined #openstack-infra14:54
fungisdague: whatcha mean write authority?14:55
sdaguewhich projects can propose a requirement change14:55
anteayardopieralski: I think you need to get fungi's attention for that request14:55
fungia change to the list of requirements, or a change to the list of projects on which the requirements list is enforced?14:55
*** rickerc has joined #openstack-infra14:56
*** krtaylor has joined #openstack-infra14:56
fungiwe've so far been letting any project be in the projects.txt file if they want, but additions/changes to global-requirements.txt can only be performed for the explicit benefit of official openstack projects14:57
*** dkranz has joined #openstack-infra14:58
*** skeppstedt has quit IRC14:58
*** mfer has quit IRC14:59
sdagueright15:00
*** todd_dsm has joined #openstack-infra15:00
sdagueok, so it looks like we actually still don't have the thing that we think we want15:00
*** mestery has quit IRC15:00
sdagueI'm reading the current requirements check job15:00
sdaguefungi: is there a way to know if a project has stable branch?15:00
mordredgit branch -a | grep stable15:01
rdopieralskifungi: would you have a moment to add the triple-o and horizon -core and -release groups to the corresponding groups on https://review.openstack.org/#/admin/projects/stackforge/tuskar-ui-extras,access ?15:01
*** kmartin_away is now known as kmartin15:02
*** MaxV has quit IRC15:03
*** david-lyle has quit IRC15:03
*** daya_k has joined #openstack-infra15:04
*** MaxV has joined #openstack-infra15:04
fungisdague: for p in `ssh -p 29418 review.openstack.org gerrit ls-projects | grep ^openstack | grep -v ^openstack/openstack$` ; do wget -qO/dev/null 'http://git.openstack.org/cgit/'$p'/tree/?h=stable/havana' && echo $p ; done15:04
*** MarkAtwood has joined #openstack-infra15:04
fungirdopieralski: sure, just a sec15:04
*** AJaeger_ has joined #openstack-infra15:04
*** markmcclain has quit IRC15:05
AJaeger_anteaya: regarding https://review.openstack.org/#/c/124593 - good catch. i thought tox -e zuul would run it in the gate but that seems not to be the case.15:05
AJaeger_Let me enhance the test...15:05
*** dims has quit IRC15:06
sdagueok, I'm going to do a thing15:06
sdagueand I think the fall out will be a bunch of people will want to leave g-r on their own15:06
fungirdopieralski: done15:06
*** dims has joined #openstack-infra15:06
fungisdague: that for loop was how i figured out which projects had a stable/havana branch (so i can delete them today)15:07
*** skeppstedt has joined #openstack-infra15:07
*** andreaf has quit IRC15:07
*** andreaf has joined #openstack-infra15:08
*** david-lyle has joined #openstack-infra15:09
AJaeger_fungi, anteaya: What about adding the test to  gate-project-config-layout? is there a specific reason that tox -e zuul does more than the layout gate?15:10
*** dims has quit IRC15:10
rdopieralskifungi: thank you very much!15:12
*** dustins has quit IRC15:12
AJaeger_fungi, seems somebody added me to project-config as core. Do you have any special requests or guidance for me?15:14
*** rdopieralski has left #openstack-infra15:14
*** gokrokve_ has joined #openstack-infra15:15
*** gokrokve_ has quit IRC15:15
*** gokrokve_ has joined #openstack-infra15:16
*** rwsu has joined #openstack-infra15:16
*** flaviof_zzz is now known as flaviof15:16
mordredsdague: why do we want people to leave g-r ?15:16
*** bswartz has joined #openstack-infra15:17
*** dustins has joined #openstack-infra15:18
*** changbl has joined #openstack-infra15:18
*** bswartz has quit IRC15:18
*** atiwari has joined #openstack-infra15:18
*** bswartz has joined #openstack-infra15:19
fungiAJaeger_: not sure why it was done that way. perhaps for simplifying the tox-driven job while keeping multiple checks running on one worker. couple perhaps be broken out into multiple new tox envs and then just call those from the zuul env currently there15:19
fungiAJaeger_: it was me who added you... http://lists.openstack.org/pipermail/openstack-dev/2014-September/047480.html15:19
*** gokrokve has quit IRC15:19
*** bswartz has quit IRC15:20
fungiAJaeger_: no specific advice, other than don't approve stuff unless you're sure it should be approved15:20
*** bswartz has joined #openstack-infra15:20
*** e0ne has quit IRC15:21
AJaeger_fungi: missed that email ;) thanks for welcoming us in!15:21
fungisdague: anteaya: ^ in case you don't have a direct e-mail feed into your brain holes, don't want the sudden extra review options to take you by surprise15:21
*** e0ne has joined #openstack-infra15:21
AJaeger_fungi: so, let me +2 for now and let others approve ;)15:21
sdagueoh hey, gerrit bot might want to put project config in here15:21
AJaeger_sdague: it should be in here already15:22
fungisdague: great idea... did we miss that?15:22
sdagueyep15:22
fungisdague: i think openstackgerrit has simply gone on walkabout again15:22
*** ChuckC_ has joined #openstack-infra15:22
sdagueoh, maybe15:22
sdaguehttps://review.openstack.org/12509515:22
sdaguemordred / fungi ^^^15:22
*** openstackgerrit has joined #openstack-infra15:24
fungiopenstackgerrit has returned to us15:24
sdague\o/15:24
rcarrillocruzzaro: hi there, i pushed a new change for the multimodule maven jobs, as the JJB repo is now in project-config. It's currently WIP, as I'd like to do some more testing but I'd appreciate if you could have a look15:25
*** hdd has quit IRC15:25
*** salv-orlando has joined #openstack-infra15:25
rcarrillocruzit now uses curl, no maven for deploying15:25
sdagueso, I think with 125095 we're going to get the enforcement that we all think we should have15:25
*** unicell has quit IRC15:25
*** chuckC has joined #openstack-infra15:25
rcarrillocruzhttps://review.openstack.org/#/c/125011/15:25
sdagueand basically all master only projects will probably be requirements wedged15:25
sdaguebut... at least it will be the thing that people seem to want15:26
*** mrmartin has joined #openstack-infra15:26
*** jcoufal has quit IRC15:27
mordredsdague: yes. I agree with that patch15:27
mordredsdague: and I do agree that some stuff may be wedged, but that's the life of something that doesn't have stable branches15:28
*** mfer has joined #openstack-infra15:30
*** skeppstedt has quit IRC15:30
*** skeppstedt has joined #openstack-infra15:31
*** stevemar has joined #openstack-infra15:32
clarkbsdague why not git branch -a in requirements for SUPPORTED_BRANCHES15:32
*** coolsvap|afk is now known as coolsvap15:32
clarkbthen we dont need to manually update multiple locations15:32
sdaguemy git branch -a includes   remotes/origin/stable/folsom15:34
sdague  remotes/origin/stable/grizzly15:34
sdague  remotes/origin/stable/havana15:34
sdague  remotes/origin/stable/icehouse15:34
sdagueso... that's why15:34
*** shashankhegde has joined #openstack-infra15:34
clarkbwe can filter that15:34
sdagueby what?15:34
clarkband do in our image builds for example15:34
*** MarkAtwood has quit IRC15:34
sdagueright, but is that any more complicated than changing 1 tupple every 6 months?15:35
*** signed8bit has joined #openstack-infra15:35
*** skeppstedt has quit IRC15:35
*** afazekas has quit IRC15:35
sdaguethis has the nice impact of being very explicit15:35
openstackgerritDenis M. proposed a change to openstack-infra/project-config: Added jobs for MongoDB and Percona datastores  https://review.openstack.org/12505715:36
stevemaranteaya, AJaeger sdague congrats y'all :)15:36
*** gyee has joined #openstack-infra15:37
*** krtaylor has quit IRC15:37
*** gokrokve_ has quit IRC15:37
*** gokrokve has joined #openstack-infra15:38
*** krtaylor has joined #openstack-infra15:38
sdagueapparently stable/icehouse devstack has been code unlandable for a month :)15:39
*** gokrokve has quit IRC15:39
*** gokrokve has joined #openstack-infra15:39
*** Daisy has joined #openstack-infra15:40
*** todd_dsm has quit IRC15:41
*** ihrachyshka has quit IRC15:42
*** salv-orlando has quit IRC15:43
pelixzaro: wondering is it worth changing the tests to treat non-existing output candidates as empty strings? Would avoid adding a number of empty files for the duplicate jobs change15:44
*** gokrokve has quit IRC15:44
AJaeger_stevemar: thanks115:44
openstackgerritAndreas Jaeger proposed a change to openstack-infra/project-config: Use tox -e zuul for layout test  https://review.openstack.org/12510115:44
*** atuvenie has quit IRC15:44
AJaeger_anteaya: 125101 is the base for the gating on the alphabetized patch in https://review.openstack.org/12459315:45
*** jgrimm is now known as zz_jgrimm15:46
clarkbsdague we have a lot of knobs to turn when adding new stable branches and one fewer should lead to fewer mistakes15:46
openstackgerritDarragh Bailey proposed a change to openstack-infra/jenkins-job-builder: Error on duplicate job names being found  https://review.openstack.org/5673415:48
*** VijayTripathi has joined #openstack-infra15:49
sdagueso I just thought about the req thing again in the shower. I think my change is a mobius strip of terribleness15:53
openstackgerritA change was merged to openstack-dev/hacking: Add a check for old style class declarations  https://review.openstack.org/11684615:53
sdaguebecause here is the actual behavior we've never decided on15:53
openstackgerritpatrick-crews proposed a change to openstack-infra/elastic-recheck: Make function for logstash query encoding  https://review.openstack.org/12224615:53
*** zz_jgrimm is now known as jgrimm15:53
sdaguestable/icehouse: foo>0.115:53
sdaguemaster: foo>0.215:53
sdaguewhat is a master only project supposed to do?15:54
sdaguebecause if the answer is stable == master15:54
sdaguethan I don't understand why g-r has branches15:54
*** dizquierdo has quit IRC15:55
sdaguebut if the answer isn't that, then is it that master only branches should actually get oldest stable syncs instead of master syncs?15:55
clarkbpip needs that dependency resolver15:55
sdaguebut that doesn't actually solve it either15:55
anteayafungi stevemar thank you15:55
sdaguebecause ... we do pip multiple times15:55
openstackgerritAndreas Jaeger proposed a change to openstack-infra/project-config: Add more checks for zuul/layout.yaml  https://review.openstack.org/12459315:55
clarkbsdague it would though. just like apt yells in those cases15:56
sdagueclarkb: no, because pip doesn't care about what you did before15:56
clarkbbut a pip dep resolver should/would15:57
sdaguewe're going to run pip 40 times15:57
clarkbbecause the dep resolver must consider the world15:57
*** shashankhegde has quit IRC15:57
clarkbhowever I am sure python would say the actual way to fix this is virtualenvs15:57
sdagueso that would mean that some ubuntu package would have a pin on a thing15:58
*** emagana has quit IRC15:58
anteayaAJaeger_: sorry I was on the phone, just getting my head around what is happening in those patches15:58
sdagueand we'd never be able to test again unless we built python from source?15:58
*** ashaeron has quit IRC15:58
sdaguebecause there is a lot of the world that we don't control15:58
*** emagana has joined #openstack-infra15:58
clarkbsdague or uninstall that thing via apt and install via pip15:58
*** yaguang has joined #openstack-infra15:58
sdagueclarkb: like some of the system level tools?15:58
clarkbbut ya virtualenvs are likely the official answer15:59
clarkbsdague yup15:59
AJaeger_anteaya: no problem....15:59
sdagueyeh... so anyway, I think we remain in terribleness on all of that15:59
clarkbwe do this with things like requests and cloud init iirc15:59
sdaguewe actually don't with requests15:59
clarkbinfra does15:59
*** markmcclain has joined #openstack-infra16:00
*** dims has joined #openstack-infra16:00
anteayafungi: so timing the creation of new projects with the creation of images16:00
*** amotoki has joined #openstack-infra16:00
*** esker has quit IRC16:00
clarkbsystem requests is used by a system lib but its too old so we update via pip16:00
sdagueok, on trusty nodes we don't16:00
anteayaso if we approve new projects, do you want us to check with you first to ensure it doesn't interfere with image creation?16:00
*** emagana_ has joined #openstack-infra16:00
sdagueand there is something else deep in that seems to drag it in, that I haven't unwound yet16:01
anteayasince those two operations clash if I recall16:01
*** dims_ has joined #openstack-infra16:01
clarkbsdague ya I think we do this on centos6 and precise16:01
*** emagana has quit IRC16:01
anteayaAJaeger_: can I test this patch on my local env? https://review.openstack.org/#/c/124593/216:02
clarkbanteaya they only clash if project creation fails16:02
clarkbanteaya I wouldnt worry about it too much16:02
anteayawhich happens randomly for undetermined reasons, yes?16:02
AJaeger_anteaya: yes, you should16:02
anteayaclarkb: k, thanks16:02
anteayaAJaeger_: k, will attempt16:03
*** jgrimm is now known as zz_jgrimm16:04
zaromorning16:04
clarkbsdague the problem with virtualenv per service is it likely wont be long before we break the distros16:04
clarkbwe could also stable branch everything16:05
sdagueclarkb: yeh... so I'm having a hard time caring a lot about that given that the distros aren't helping with the problem at all16:05
*** zaphod42 has joined #openstack-infra16:06
*** sabeen3 has joined #openstack-infra16:06
*** dims has quit IRC16:06
*** sabeen has quit IRC16:06
*** HeOS has quit IRC16:06
AJaeger_anteaya: i think we should first integrate dhellmann's patch and then go test 124593...16:08
*** tothanhcong has joined #openstack-infra16:08
*** gokrokve has joined #openstack-infra16:08
*** tothanhcong has quit IRC16:08
AJaeger_so, to untangle this, could somebody help reviewing and approving https://review.openstack.org/124759 and https://review.openstack.org/124517, please?16:09
*** tothanhcong has joined #openstack-infra16:09
fungianteaya: adding new projects doesn't really break image creation unless the project addition itself breaks, or unless you get really lucky and it merges at just the wrong time16:10
zarohashar: need help with 12488516:10
anteayafungi: awesome16:10
*** yamahata has quit IRC16:11
*** eglynn-office is now known as eglynn-officeafk16:11
*** tothanhcong has quit IRC16:11
anteayaAJaeger_: yes, the order of the patches seems to be important here in order to tox to do its thing16:11
*** ildikov_ has quit IRC16:11
anteayamorning zaro16:11
*** zz_jgrimm is now known as jgrimm16:12
*** koolhead17 has quit IRC16:12
anteayaAJaeger_: did you want to merge dhellmann's patch first rather than work on the dependency dance?16:12
zaropelix: hey, welcome back. jjb release this week.  is there something you really like to be included?16:12
AJaeger_anteaya: yes, let's get dhellmann's in (thanks sdague and clarkb for reviews!) and then I'll sort it out.16:12
AJaeger_Let me add some WIPs to mine ;)16:13
hasharzaro: yeah I wasn't sure how to write the tests myself. I am in audio right now16:13
fungianteaya: probably the bigger thing which causes interruptions is adding new channels to eavesdrop/meetbot since that can interrupt meeting logging and leave stale topics, but conveniently we didn't move that one into project-config since it's handled as an array in a puppet manifest16:13
*** congtothanh has joined #openstack-infra16:13
openstackgerritDarragh Bailey proposed a change to openstack-infra/jenkins-job-builder: Error on duplicate job names being found  https://review.openstack.org/5673416:13
*** yfried|afk has quit IRC16:13
*** gokrokve has quit IRC16:13
pelixzaro: initial python3 compatibility? Seems like should try and get in place?16:13
clarkbAjaeger_ dont forget the parent of 12475916:14
*** packet has joined #openstack-infra16:14
anteayafungi: yay16:14
openstackgerritA change was merged to openstack-infra/project-config: Do not run unit test jobs for oslo-specs  https://review.openstack.org/12451716:14
*** packet is now known as Guest5592116:14
zarohashar: hmm, too bad msabram isn't around.16:15
zaropelix: ok.  lets do it. which should i review?16:16
AJaeger_clarkb: I mixed up the ID of parent of 124759 but you all sorted it out - thanks a lot!16:17
hasharzaro: I might have some free time tonight. Will play a bit with mock :]16:18
anteayaAJaeger_: I need to have some lunch, let's see what happens with merge check on your patch16:18
pelixzaro: just verifying after making the changes you recommended16:18
openstackgerritDarragh Bailey proposed a change to openstack-infra/jenkins-job-builder: Make JJB python 3 compatible  https://review.openstack.org/11870316:19
*** r-daneel has joined #openstack-infra16:19
clarkbis ^ an unrevert?16:19
*** yfried|afk has joined #openstack-infra16:19
clarkbdid we sort out what broke?16:19
*** baoli has quit IRC16:19
*** congtothanh has quit IRC16:19
openstackgerritA change was merged to openstack-infra/project-config: Remove oslotest cross-project unit test jobs  https://review.openstack.org/12475816:19
openstackgerritA change was merged to openstack-infra/project-config: Fix up tests after repo reorganization  https://review.openstack.org/12475916:20
*** r-daneel has quit IRC16:20
*** MaxV has quit IRC16:21
*** IvanBerezovskiy has left #openstack-infra16:21
clarkbsdague would probably be reasonable to have a discussion around venv per service. dstufft is certainly in favor and tripleo does it for sanity16:21
sdagueyeh16:21
sdaguemaybe that should be the ML thread :)16:21
clarkbya probably16:22
clarkbor even cross project summit session16:22
mordredclarkb: why?16:22
mordredlike, to solve which problem?16:22
mordredhang on - lemme read the whole scrollback16:23
sdaguemordred: yeh, go back to mobius strip of terribleness16:23
*** Ryan_Lane has quit IRC16:23
*** bradjones has quit IRC16:24
mordredok. so - before we unwind all of the things16:24
mordredthe original question you posed was "sdague | what is a master only project supposed to do?"16:24
zaroanybody else concerned that our docs are not getting updated in rtfd.org?16:25
mordredand the idea is that a master only project is supposed to still work with stable releases16:25
mordredzaro: no16:25
*** Longgeek has quit IRC16:25
*** unicell has joined #openstack-infra16:25
mordredso regardless of how pip works or how virtualenvs or how g-r works16:25
mordredwe need to verify that master of python-novaclient still works with stable/icehouse16:25
zarook.  should we continue to upload to rtfd then?16:25
*** bradjones has joined #openstack-infra16:26
*** bradjones has quit IRC16:26
*** bradjones has joined #openstack-infra16:26
mordredand since dib is a thing that is following the same model, we need to also check that master dib works in a stable/icehouse build16:26
*** shashankhegde has joined #openstack-infra16:26
mordredwe can rework all of the enforcement mechanics all day long, but we do need to verify those things16:26
zaroi mean there's a doc-on-rtfd that tons of jobs are using, but it doesn't seem to work :(16:27
*** mrmartin has quit IRC16:27
sdaguemordred: ok, so we do novaclient vs. stable/icehouse16:27
mordredoff the cuff, I do not believe that venv-per-service has anything to do with the above positive or negative - it's a thing we might consider doing, and I do not think it will solve anything in this discussion16:27
mordredsdague: that's right16:28
*** r-daneel has joined #openstack-infra16:28
mordredand we don't actually co-gate dib with the integrated release, so we don't actually need to do a devstack test on it16:28
fungizaro: i have doubts that's what's broken. rtfd is also supposed to update itself daily, independent of that hook16:28
*** jistr has quit IRC16:28
*** thedodd has quit IRC16:28
*** Guest55921 has quit IRC16:28
fungizaro: if projects aren't getting their docs updated there at all, then that points to a shift in rtfd's model/configuration or else perhaps rtdf itself has broken16:29
sdaguemordred: so in that case, why do we do a requirements check?16:29
fungitfdr... some combination of letters anyway16:29
mordredbut if the model holds for requirements from the client libs, then we should backport the dib-utils split to stable/icehouse g-r, and the porting of that patch should be enough to verify that installing master dib alongside icehouse does not break the world16:29
*** zns has quit IRC16:29
*** Guest68510 has quit IRC16:30
mordredsdague: we do the requirements check to make sure that the contents of the requirements files describe reality and work16:30
sdagueso it seems like what we're actually saying16:30
sdague"we should test that dib can work in an openstack environment"16:30
sdaguewe don't do that16:30
*** mpaolino has quit IRC16:30
mordredIm not saying that16:30
mordredat all16:30
sdagueok16:30
*** zns has joined #openstack-infra16:31
mordredI'm saying that the addition of dib-utils to global-requirements, because dib is a master-only project, should have a corresponding change to add dib-utils to g-r:stable/icehouse. if that patch breaks something, then we should not land the corresponding one to master16:31
mordredwe are not testing that right now, but that is the consequences of having master-only projects at all16:31
mordredand it holds for all of them16:31
sdagueexcept, what's requiring dib-utils ?16:32
mordreddib16:32
sdagueand what's requring that?16:32
mordredit's a top level thing16:32
*** AzherKhan has joined #openstack-infra16:32
mordredalso, ironic16:32
sdagueit's not an integrated release thing16:32
mordredthe ironic devstack tests use dib16:32
derekh--help16:32
fungiderekh: unrecognized option '--help'16:33
openstackgerritJoe Gordon proposed a change to openstack-infra/elastic-recheck: Don't show ER patches under open reviews  https://review.openstack.org/12511716:34
*** amcrn has joined #openstack-infra16:34
derekh-?16:34
derekhfungi: hmm, that didn't work either16:34
fungiSegmentation Fault: core dumped16:34
sdagueso... this means that when evaluating whether we add a requirement to master, do we take into consideration the possibly disruptive impact on stable of adding new requirements?16:34
derekh;-)16:34
*** koolhead17 has joined #openstack-infra16:34
mordredsdague: we have to if we're adding the new requirement because of a master-only project16:35
mordredintegrated/not-integrated aside for a sec - it's the same thing with the recent splits of keystone middleware stuff16:35
zarofungi: so i guess we would need help from rtfd maintainers to debug?  do you know who to ask?16:36
sdagueso master only projects that aren't part of the integrated release, get to add requirements to openstack that people need to backport package16:36
mordredOR they don't16:36
sdaguethat's the part that has always seemed odd to me16:36
mordredit just needs to be considered16:36
mordredit's possible that the answer is "no, that would break stable"16:36
mordredin which case they can suck it16:37
sdagueok, so, how do we know if they need to suck it?16:37
mordredpropose the backport of the patch to stable/icehouse and see if anything breaks16:37
sdaguethat means ... we actually want to check master against oldest stable req right?16:37
*** atuvenie has joined #openstack-infra16:37
clarkbmordred venvs per project makes us care less about master foo working with icehouse bar because the coexist in venvs. its a bit of a heavyweighr solution yes and likely to cause distros pain16:38
*** baoli has joined #openstack-infra16:38
mordredI was just trying to think through the combinations and I'm not 100% sure there is a direct gate we can build16:38
pleia2good morning16:38
mordredclarkb: that's right. and so far we've been considering the distros an important consumer16:38
mordredso I would not be willing to be party to that change without a TC discussion16:38
sdaguemordred: yeh, especially with this idea that icehouse is getting an extra 6 months of support16:39
sdaguewe're going to be juggling 4 branches at once16:39
clarkbwas just going to mention that16:39
*** patrickeast has joined #openstack-infra16:40
mordredwe could completely ditch g-r and move to a completely independent list per-service16:40
mordredbut that's a pretty substantial change in thinking16:40
sdagueif we had venvs16:40
mordredright16:40
sdagueyep16:40
mordredI'm just saying - that's fairly massive. it may be the right choice16:40
mordredbut it's fairly massive16:40
sdaguesure16:40
openstackgerritSean Dague proposed a change to openstack-infra/project-config: enforce stable requirements  https://review.openstack.org/12509516:40
mordredit will also ABSOLUTELY screw the distros16:40
sdagueok, so that's conceptually the thing16:40
mordredalmost immediately16:40
fungiit also means we're saying "this is how you cd openstack now if you're not using distro packages"16:41
mordredbecause we know how divergent we were in the projects per-convergence16:41
clarkbyup16:41
sdaguewell, looking at how they patch the f* out of a lot of these python packages16:41
*** hdd has joined #openstack-infra16:41
* mordred is fine with that16:41
mordredbtw16:41
fungiand the docs team is just going to love us if we do that16:41
sdagueyeh, anyway, I have to go get food16:41
mordredas all of the friday discussions around current state of distros16:41
mordredbut we can't just make that call here16:41
sdaguemordred: right 100% agreed16:42
fungisummit session +=16:42
sdagueI'm just trying to figure out if there is a sane way through this on the requirements side without that16:42
clarkbthe other alternative is stable branch everything16:42
mordredsdague: what's wrong, for now, with proposing a backport of dib-utils to stable/icehouse16:42
sdaguewhich is why I'm challenging various things, hoping someone will just say "no silly, it works fine if yuo do A"16:42
*** hashar has quit IRC16:42
mordreddoing that a few times by hand to see if patterns emerge16:42
sdaguemordred: we do that all the time16:43
mordredand then trying to encapsulate what we learn in a suggestion?16:43
fungisdague: pretty sure none of it works fine, ever16:43
sdaguemordred: and it feels really weird every time we do to me16:43
fungiour choice is between a variety of different sorts of broken compromises16:43
sdaguefungi: right16:43
mordredyah16:43
mordredremember we're trying to have our cake and eat it too all the time16:43
sdagueok, lunch16:43
clarkbI think if we want to make long term support of releases a thing and support distros everything needs a stable branch16:43
fungimmmcake16:44
mordredI think less things need stable branches, not more :)16:44
* mordred says screw the distros and screw stable branches - which, btw, is what the original plan was before the distros showed up and whinned in boston16:44
clarkbthen we shouldnt support things for 18 months post release16:45
*** lttrl has quit IRC16:45
fungiwell, it's 15 months post release (final icehouse point release scheduled for july)16:45
*** _nadya_ has joined #openstack-infra16:45
mordredI keep getting told that people should get openstack from their vendors and not from us -that it's not our job to produce a product16:45
mordredif we accepted that and adopted linus' thinking, we'd stop supporting anything and let ubuntu and redhat and piston all sort it out16:46
*** Daisy has quit IRC16:46
*** Daisy has joined #openstack-infra16:46
fungiwell, linus' model does have stable branches, fwiw16:46
mordredI would advocate ditching releases as well though and just going to a completely CD-oriented pipeline16:46
*** yfried|afk has quit IRC16:46
SpamapSmordred: awake now. Whats up?16:47
mordredSpamapS: dib screwed stable/icehouse16:47
mordredSpamapS: and it has caused us all to go all meta about releases16:47
*** doude has quit IRC16:47
zarojhesketh: still working on this? https://review.openstack.org/#/c/107724/ ?16:47
mordredSpamapS: but - short version, dib added dib-utils in master g-r and so it doesn't work in stable/icehouse16:47
annegentlehonestly mordred I'd be happy to drop the install guide like a hot potato16:47
mordredannegentle: I second that motion16:48
SpamapSmordred: because stable/icehouse tests using dib master?16:48
annegentlebut board members approached me when I vented about it a few summits ago16:48
mordredSpamapS: that's right - because there is no stable/icehouse of dib16:48
SpamapSmordred: there are pypi releases...16:48
fungiannegentle: we've got all new board members now ;)16:48
annegentlebut if the time is right after this release let's ditch16:48
SpamapSmordred: so... <= in requirements.txt ?16:48
mordredSpamapS: that is irrelevant16:48
annegentleheh16:48
annegentlenice16:48
annegentleI like our board! and the members that approached me were super supportive16:48
*** _nadya_ has quit IRC16:49
*** dims_ has quit IRC16:49
mordredSpamapS: I believe we use master of dib in devstack ... although I suppose you're suggesting that we just pip install dib?16:49
mordredsdague: ^^ ?16:49
SpamapSmordred: can't we just not pull things without stable branches via git?16:49
annegentleso that's all cool. but honestly, looking at the web stats, the most widely read one is ubuntu and that's cuz they don't have any other doc16:49
fungiwell, we're also managing to sneak more and more tc members into board seats. it's only a matter of time until they become the majority voice ;)16:49
annegentlefungi: hee16:49
*** hashar has joined #openstack-infra16:49
*** dims has joined #openstack-infra16:49
mordredsdague: we may have a good specific suggestion here ....16:49
mordredannegentle: did any of those board members offer doc writers to help?16:50
annegentlemordred: yes they did16:50
mordredoh. ok then16:50
annegentlemordred: so yeah, money where mouth16:50
mordredif people are ponying up humans to work on a thing, I'm much more in support of supporting that thing16:50
*** thedodd has joined #openstack-infra16:50
annegentlemordred: oh I'm meeting with the Eucalyptus writing team this afternoon!16:51
*** otherwiseguy has quit IRC16:51
annegentlemordred: not that they want to work on install docs :)16:51
annegentlemordred: but it's cool to know those writers16:51
*** markmcclain has quit IRC16:52
*** MarkAtwood has joined #openstack-infra16:52
*** shashankhegde has quit IRC16:52
mordredannegentle: yah - I have a hunch we're all going to get to know them better ...16:52
*** tonytan4ever has quit IRC16:52
anteayapleia2: how were penquins and wolverines?16:52
*** scottda has joined #openstack-infra16:53
*** sarob has joined #openstack-infra16:53
*** bhuvan has joined #openstack-infra16:54
*** bodepd_ has quit IRC16:54
mordredsdague, SpamapS: https://review.openstack.org/12512116:54
mordredlike that?16:55
clarkbok I am finally on a real computer now16:55
*** mwagner_lap has quit IRC16:55
*** amuller_ has joined #openstack-infra16:55
*** _nadya_ has joined #openstack-infra16:55
mordredclarkb: congratulations16:55
pleia2anteaya: they were a lot of fun16:55
*** laxity has quit IRC16:56
anteayaclarkb: are you at your portland home, with furniture and a computer?16:56
anteayapleia2: \o/16:56
openstackgerritDarragh Bailey proposed a change to openstack-infra/jenkins-job-builder: Error on duplicate job names being found  https://review.openstack.org/5673416:56
clarkbanteaya: yup. sitting on couch with laptop. The desk in office still needs to be put cobbled together. I need more wireless antennas16:56
*** laxity has joined #openstack-infra16:57
*** amuller__ has joined #openstack-infra16:57
*** emagana_ has quit IRC16:57
* fungi imagines clarkb wandering about with various wireless antennae strapped on16:57
*** ihrachyshka has joined #openstack-infra16:58
anteayaclarkb: yay for furtiture, in a house/home16:58
*** amuller_ has quit IRC16:58
*** amuller has quit IRC16:58
*** bodepd has joined #openstack-infra16:58
*** derekh has quit IRC16:58
*** amuller_ has joined #openstack-infra16:58
*** otherwiseguy has joined #openstack-infra16:58
anteayafungi: quite the visual there16:58
*** bodepd has quit IRC16:59
*** mwagner_lap has joined #openstack-infra16:59
*** marun has joined #openstack-infra16:59
*** hashar has quit IRC16:59
*** bodepd has joined #openstack-infra17:00
*** sandywalsh_ has quit IRC17:01
*** krtaylor has quit IRC17:01
*** esker has joined #openstack-infra17:01
*** armax has joined #openstack-infra17:01
*** zns has quit IRC17:02
*** _nadya_ has quit IRC17:02
*** amuller__ has quit IRC17:02
*** sandywalsh has joined #openstack-infra17:03
anteayawho is chairing today's infra meeting?17:03
anteayaI was going to tidy the agenda17:03
*** amuller_ is now known as amuller17:03
openstackgerritDarragh Bailey proposed a change to openstack-infra/jenkins-job-builder: Make JJB python 3 compatible  https://review.openstack.org/11870317:04
*** bhuvan_ has joined #openstack-infra17:04
morgan_remote_Is anything weird going on with gerrit? I seem to be stuck in a login loop. Might just be mobile browser but it was working last night.17:04
fungianteaya: i am, and was about to go do that--i have a couple things i need to make sure i add anyway17:04
*** jedimike has quit IRC17:05
clarkbmorgan_remote_: not that I have seen17:05
morgan_remote_I login and it redirects me to ubuntuone, redirects to gerrit then to Ubuntu one and requires login again. Huh weird.17:05
morgan_remote_And does that endlessly.17:05
* clarkb tries logging out and back in again17:06
clarkbmorgan_remote_: no problem here. could your browser be caching stuff oddly?17:06
*** markmcclain has joined #openstack-infra17:06
*** bhuvan has quit IRC17:06
fungimorgan_remote_: is your mobile browser an ubuntuphone?17:07
morgan_remote_No. iPhone17:07
*** atuvenie has quit IRC17:07
*** tonytan4ever has joined #openstack-infra17:07
morgan_remote_Will try clearing out cache etc17:07
*** Daisy has quit IRC17:08
*** arosen-home2 has joined #openstack-infra17:08
SpamapSmordred: left a comment17:08
fungijust figured it would be amusing if ubuntuphone had issues with ubuntu's openid17:09
*** thedodd has quit IRC17:09
morgan_remote_clarkb: figured it out. 3rd party cookie blocking was turned on.17:09
*** gokrokve has joined #openstack-infra17:09
* clarkb syncs project-config finally17:11
*** kgavini has joined #openstack-infra17:11
anteayafungi: I will get out of your way and do something else17:11
* anteaya does something else17:11
*** emagana has joined #openstack-infra17:12
*** jgallard has quit IRC17:13
*** shashankhegde has joined #openstack-infra17:13
*** lttrl has joined #openstack-infra17:14
*** gokrokve has quit IRC17:14
*** esker has quit IRC17:14
*** bradjones has quit IRC17:15
*** MaxV has joined #openstack-infra17:16
* clarkb adds dib to the agenda if it isn't already there17:16
*** gokrokve has joined #openstack-infra17:16
openstackgerritA change was merged to openstack-infra/project-config: heat-dsvm-functional INSTALL_TESTONLY=1  https://review.openstack.org/12423317:17
*** mrmartin has joined #openstack-infra17:17
clarkboh good its still there udner priority efforts17:17
*** zns has joined #openstack-infra17:17
*** mmaglana has joined #openstack-infra17:17
*** harlowja_away is now known as harlowja17:18
anteayaAJaeger_: I think you can un-WIP this one now: https://review.openstack.org/#/c/125101/ I can get tox to work on my local environment17:20
zaroclarkb: i don't think jjb compare-xml test is working anymore.  do you know if project-config is at a point where compare-xml should be working yet?17:20
krotscheckSo, I have two reviews that undo-and-fix my CORS work from yesterday, https://review.openstack.org/#/c/124934/ and https://review.openstack.org/#/c/124936/ - anyone able to look at those for me? (The underlying issue is that the scp publisher does weird path things that break things)17:20
clarkbzaro: I am not sure. anteaya and AJaeger_ were working on similar things earlier though. They may know17:21
*** amotoki has quit IRC17:22
*** praneshp has joined #openstack-infra17:23
annegentlesdague: around?17:23
clarkbkrotscheck: I dont claim to fully understand the gruntfile17:23
anteayazaro: I think that tests on project-config either should be working now or we should be in the process of fixing what isn't working17:23
anteayazaro: how do we go about fixing compare-xml to make it work? I don't know that job very well17:24
clarkbkrotscheck: but it seems simple enough17:24
clarkbcopy:draft vs copy:dist to put files in different places17:24
*** esker has joined #openstack-infra17:25
krotscheckclarkb: Right, so the way grunt plugins work is that you can declare subcommands. So grunt copy:dist can be invoked, and grunt copy:draft can be invoked, and if you want to run both of them you can just call grunt copy.17:25
*** bhuvan_ has quit IRC17:25
openstackgerritDarragh Bailey proposed a change to openstack-infra/jenkins-job-builder: Switch to new project-config repo  https://review.openstack.org/12512417:25
pelixseems that updating the scripts in jenkins-job-builder to use the new repo for the job configs was forgotten with the reorganization17:25
krotscheckclarkb: Now that we have two distinct copy subcommands though, I have to be specific about which one is invoked during which stage of the build. So I only need copy:draft when build:draft is invoked.17:25
krotscheckclarkb: That17:26
*** bhuvan has joined #openstack-infra17:26
pelixzaro: ^^ we probably want that change in before allowing anything else through just to make sure we aren't breaking anything obvious17:26
krotscheckclarkb: That’s why I had to modify other invocations of “copy” to say “copy:dist"17:26
clarkbkrotscheck: gotcha. +217:26
krotscheckclarkb: And that’s basically it.17:26
AJaeger_zaro: are you running "tox -e zuul" locally?17:26
krotscheckclarkb: Thanks :D17:26
anteayapelix: what scripts were forgotten?17:27
AJaeger_zaro, anteaya: it should work now with current HEAD17:27
anteayaAJaeger_: the compare-xml job?17:27
pelixanteaya: yeap17:27
pelixthe compare-xml script17:27
*** bradjones has joined #openstack-infra17:27
*** flaviof is now known as flaviof_zzz17:27
*** bradjones has quit IRC17:27
*** bradjones has joined #openstack-infra17:27
clarkbfungi: https://review.openstack.org/#/c/119936/ would be a good one to get in if you have a chance to review it17:28
*** todd_dsm has joined #openstack-infra17:28
* AJaeger_ double checks compare-xml17:28
anteayapelix: where was the compare-xml script17:28
*** sacharya has joined #openstack-infra17:28
pelixanteaya: in the JJB git repo17:28
AJaeger_anteaya: project-config/tools/run-compare-xml17:28
pelixtools/run-compare-xml.sh17:28
anteayaah in tools17:28
zaropelix: agreed, but i did run it manually with change 56734 and it looks good.17:28
anteayaI wasn't asked to move it17:29
AJaeger_pelix: in JJB?17:29
anteayacan folks talk in complete sentences please?17:29
zaroanteaya: https://review.openstack.org/#/c/10400417:29
anteayaI am having trouble following the point17:29
zaroanteaya: oops, i meant http://logs.openstack.org/34/56734/12/check/jenkins-job-builder-coverage/8689ed8/17:29
*** baoli has quit IRC17:30
AJaeger_zaro, I'm still confused - what is the relevance of the log?17:30
clarkb(I am flushing project-config changes now)17:30
anteayaclarkb: flushing them from what?17:30
zaroanteaya, AJaeger_ : sorry, wrong link again.  this one is what i really meant. http://logs.openstack.org/34/56734/12/check/jenkins-job-builder-compare-xml/1b97efb/17:31
pelixAJaeger: yes in jenkins-job-builder git repo. It's used by the jenkins-job-builder-compare-xml job to catch if any changes to JJB are going to cause problems with the existing openstack JJB configs, example of the problem http://logs.openstack.org/03/118703/4/check/jenkins-job-builder-compare-xml/4553e48/console.html17:31
zaroit's the compare-xml test.17:31
*** baoli has joined #openstack-infra17:31
anteayapelix: so you are saying that a script that was never identifying to be moved into the project-config repo is not in the project-config repo17:31
clarkbanteaya: flushing out changes that are ready to merge17:31
anteayaclarkb: ah okay, thanks17:31
clarkbanteaya: I am treating older +1s from our new core members as +2s and approving things17:31
*** ZZelle_ has joined #openstack-infra17:31
anteayaclarkb: kk, thank you17:31
AJaeger_clarkb: thanks!17:31
pelixanteaya: no, I'm saying it was referencing the old config repo to grab the yaml configs that were moved out into project-config and the script was never updated to point to the new repo.17:32
*** tonytan4ever has quit IRC17:32
*** che-arne has quit IRC17:32
openstackgerritA change was merged to openstack-infra/config: Removes zuul/layout-dev.yaml  https://review.openstack.org/12489817:32
*** yfried|afk has joined #openstack-infra17:32
pelixDon't think the script should have been moved, just that it wasn't updated to take into account the changes that occurred.17:32
anteayapelix: ah okay now I understand thank you, yes if the job points to the content which isnt' there, I can see it breaking17:32
*** AzherKhan has quit IRC17:33
*** Ryan_Lane has joined #openstack-infra17:33
anteayapelix: yes, can we edit that to point to where the content is now then?17:33
*** AzherKhan has joined #openstack-infra17:33
zaroanteaya, AJaeger_ : the checkout is done in tools/run-compar.sh it needs to point to new project.17:33
pelixanteaya: see https://review.openstack.org/12512417:33
zaroanteaya, AJaeger_ : you guys want to fix? or i can do17:33
openstackgerritA change was merged to openstack-infra/project-config: Run a shell script instead of tox for osc functional  https://review.openstack.org/12427617:34
AJaeger_pelix, zaro: There's both a run-compare in jjb and in project-config. I was thinking you talk about project-config but you meant the jjb one.17:34
anteayapelix: +1, thank you17:34
anteayazaro: looks like pelix just needs your review on 12512417:35
*** coolsvap is now known as coolsvap|afk17:35
anteayapelix: is that accurate?17:35
AJaeger_clarkb, fungi could you review https://review.openstack.org/#/c/125124/ to fix jjb gating, please?17:35
pelixyep, it appears to fix the job :)17:35
anteayayay17:35
openstackgerritA change was merged to openstack-infra/project-config: Run tripleo tests on horizon  https://review.openstack.org/12433317:36
* AJaeger_ is not confused anymore...17:36
openstackgerritAndreas Jaeger proposed a change to openstack-infra/project-config: Group stackforge projects in layout.yaml  https://review.openstack.org/12458917:36
clarkbAJaeger_: looking17:36
openstackgerritAndreas Jaeger proposed a change to openstack-infra/project-config: Add more checks for zuul/layout.yaml  https://review.openstack.org/12459317:36
openstackgerritAndreas Jaeger proposed a change to openstack-infra/project-config: Use tox -e zuul for layout test  https://review.openstack.org/12510117:36
* AJaeger_ was working on the three patches above - that also include run-compare ;917:36
zaropelix: ohh, you are a step ahead!17:37
*** yaguang has quit IRC17:37
clarkb125124 lgtm17:39
*** habib__ has quit IRC17:39
clarkbI checked the test output too17:39
AJaeger_thanks, clarkb17:39
*** mflobo_ has joined #openstack-infra17:40
*** linuxmohan has joined #openstack-infra17:40
openstackgerritA change was merged to openstack-infra/project-config: Gate Gnocchi against py34-postgresql  https://review.openstack.org/12434017:41
*** VijayTripathi has quit IRC17:41
SpamapSmordred: note that diskimage-builder and tripleo have developed fairly sophisticated (but not complex) tooling to allow packages vs. git ... seems like devstack might need the same.17:41
*** emagana has quit IRC17:41
mordredSpamapS: that might be a nice patch to devstack ...17:42
*** emagana has joined #openstack-infra17:42
*** tonytan4ever has joined #openstack-infra17:42
mordredSpamapS: because I know that testing current pip python*client is desirable as is testing git17:42
mordredsdague: ^^ or?17:42
*** daya_k has quit IRC17:42
openstackgerritA change was merged to openstack-infra/config: Trim jenkins::slave package installations further  https://review.openstack.org/11993617:43
*** _nadya_ has joined #openstack-infra17:43
clarkbfungi: ty17:43
SpamapSmordred: we don't want to fall too far down the testing-matrix-all-the-things hole, but having at least the extreme edges of the matrix handled does seem a worthy goal.17:43
*** mflobo has quit IRC17:43
*** mflobo__ has joined #openstack-infra17:43
mordredSpamapS: well, if there is solid tooling, that seems like a good thing to have - whether we exercise that tooling in all possiblity permutations is orthogonal17:44
*** emagana_ has joined #openstack-infra17:45
*** emagana has quit IRC17:45
*** gokrokve has quit IRC17:45
*** mflobo_ has quit IRC17:46
*** gokrokve has joined #openstack-infra17:46
*** arosen-home2 has quit IRC17:46
*** arosen1 has joined #openstack-infra17:46
*** arosen1 has quit IRC17:46
*** arosen1 has joined #openstack-infra17:47
pelixzaro: ;)17:47
zaroAJaeger_: you pointed out something that still confuses me, why run-compare-xml.sh in both project-config and jjb.  i'm not sure one in project-config is used, might be opportunity to clean up?17:47
*** arosen1 has quit IRC17:47
SpamapSmordred: sure. I am thinking we should just basically turn your patch into an if on an environment variable or conf setting and keep that conf setting extremely coarse grained.. like    if [ -n $install_non_openstack_things_from_pip ] ; then ...17:47
*** arosen1 has joined #openstack-infra17:48
SpamapSnon_integrated things might be a better term17:48
AJaeger_zaro: We run it in project-config to see what has changed from one commit to another. We're looking at changes in project-config17:48
*** linuxmohan has quit IRC17:48
AJaeger_zaro: it's run in jjb to check that there are no regressions with your patch to jjb17:48
*** sdake_ has joined #openstack-infra17:48
*** rushiagr is now known as rushiagr_away17:48
*** arosen1 has quit IRC17:49
*** gokrokve has quit IRC17:49
*** linuxmohan has joined #openstack-infra17:49
*** gokrokve has joined #openstack-infra17:49
*** Sukhdev has joined #openstack-infra17:51
*** hashar has joined #openstack-infra17:51
*** rushiagr_away is now known as rushiagr17:51
*** sabeen has joined #openstack-infra17:52
zaroAJaeger_: ahh looks like a new test that wasn't in config.17:52
AJaeger_zaro, it was in config as well...17:53
*** reed has quit IRC17:53
*** cnesa15 has joined #openstack-infra17:53
openstackgerritA change was merged to openstack-infra/jenkins-job-builder: Switch to new project-config repo  https://review.openstack.org/12512417:53
*** reed has joined #openstack-infra17:53
zaroAJaeger_: was it named differently when in config?  i didn't see it.17:53
*** _nadya_ has quit IRC17:54
*** _nadya_ has joined #openstack-infra17:54
zaroclarkb: i think you should take a look at this one, https://review.openstack.org/#/c/118703/17:55
*** pelix has quit IRC17:55
*** sabeen3 has quit IRC17:55
*** dangers is now known as dangers_away17:56
AJaeger_zaro: see https://review.openstack.org/#/c/114933/ - and toggle ci to see the job was there as well17:56
AJaeger_"config-compare-xml Jenkins XML output has changed. in 2m 56s (non-voting)"17:56
zaroAJaeger_: ohh, i see. i think it's actually only run in post pipeline.  i only see it in merged changes.17:59
AJaeger_114933 was not merged17:59
*** todd_dsm has quit IRC17:59
*** arosen-home1 has joined #openstack-infra18:00
clarkbzaro: it should be rechecked once the xml comparison job is fixed18:01
*** zaphod42_ has joined #openstack-infra18:01
arosen-home1How do i become part of an additional group in gerrit? For example congress-release https://github.com/openstack-infra/project-config/blob/master/gerrit/acls/stackforge/python-congressclient.config#L618:01
zaroAJaeger_: then i'm confused.  i don't see it in any of the open config changes. what determines when it gets run?18:01
*** MarkAtwood has quit IRC18:01
clarkbarosen-home1: two ways. if the group already has members they can add you. if there are no current members gerrit admins (basically infra core) can add the initial member18:02
*** MarkAtwood has joined #openstack-infra18:02
AJaeger_zaro: it's not run since project-config was split up - you only see it in jobs before the split like 11493318:03
*** johnthetubaguy is now known as zz_johnthetubagu18:03
anteayapleia2: can you take a moment and help me understand this patch better? https://review.openstack.org/#/c/124569/118:04
*** zaphod42 has quit IRC18:04
*** sarob has quit IRC18:04
zaroAJaeger_: ok. that clears it up. thanks.18:04
*** david-lyle is now known as david-lyle_afk18:05
arosen-home1clarkb:  yea looks like the group doesn't exist yet :/.  Do i make the request through the infra-mailing list? Or i wonder if i should change the group to be congress-core ?18:05
*** hashar has quit IRC18:05
clarkbcongress-release exists18:05
* arosen-home1 which does exist. 18:05
arosen-home1clarkb:  anyone a part of it?18:05
clarkband no we usually do not want to mix those two groups18:05
clarkbarosen-home1: nope. Who should be the initial member? I can add that person18:05
*** IAX has joined #openstack-infra18:05
pleia2anteaya: sure18:05
arosen-home1Can you make me that person ?18:06
*** sdake_ has quit IRC18:06
anteayapleia2: so basically it is using shell script to add a new conditional18:06
arosen-home1arosen is my launchpad id.18:06
clarkbarosen-home1: sure18:06
arosen-home1clarkb: thanks!18:06
*** todd_dsm has joined #openstack-infra18:06
anteayapleia2: and then it filters on each conditional, taking a different action based on the filters18:06
clarkbarosen-home1: done. note we like to keep the release groups small to avoid having lots of people that can push tags18:06
clarkbarosen-home1: since mispushed tags can be a giant pain to clean up18:06
arosen-home1clarkb:  makes sense.18:06
anteayapleia2: and errors if the filters have no content18:06
anteayapleia2: yes?18:07
*** rmcall has joined #openstack-infra18:07
*** otherwiseguy has quit IRC18:07
anteayapleia2: and the action is the parsing of content to arrive at a name value for the MODULE variable18:08
*** _nadya_ has quit IRC18:08
pleia2anteaya: so it checks to see whether we're using Modulefie (deprecated, see deprecation warning at: https://docs.puppetlabs.com/puppet/latest/reference/modules_publishing.html) or metadata.json (yay!)18:08
*** gokrokve has quit IRC18:08
*** annegent_ has joined #openstack-infra18:09
pleia2but yes, as you describe is correct18:09
anteayapleia2: great thanks18:09
*** gokrokve has joined #openstack-infra18:09
sdaguemordred: ok, so that's a piece of things. However we also need to drop it from projects.txt after that, right?18:09
anteayapleia2: +2, thanks for taking the time to explain that to me18:10
*** gokrokve has quit IRC18:10
mordredsdague: no, it can still participate in projects.txt after that and it should have no ill effects on our gate18:10
pleia2anteaya: sure thing, you mostly got it yourself though :)18:10
*** gokrokve has joined #openstack-infra18:10
mordredthe problem is that it was in projects.txt AND we were processing its git repo, which means that projects.txt syncing was going on with it18:11
*** jlibosva has quit IRC18:11
pleia2anteaya: does also bring up the fact that we should switch to metadata.json at some point, but deprecated means we have some time ;)18:11
anteayapleia2: having you here helped me18:11
sdaguemordred: really?18:11
sdagueI thought the problem was it was in projects.txt18:11
*** sdake_ has joined #openstack-infra18:11
sdaguein stable/icehouse18:11
anteayapleia2: ah the cushy pillow to fall back on18:11
sdagueso the pbr try to install things script fails18:12
mordredsdague: that's only a problem if we are also processing its git repo in devstack18:12
*** _nadya_ has joined #openstack-infra18:12
*** amuller has quit IRC18:12
mordredsdague: there are tons of things in projects.txt taht we do not test or care about18:12
mordredsdague: dib was in a weird middle ground18:12
mordredis18:12
mordredsdague: where we install it from git as part of a normal devstack run18:12
mordredwhich we do not do with other things in projects.txt that we don't care about18:13
*** todd_dsm has quit IRC18:14
*** yolanda has quit IRC18:15
sdaguemordred: yeh, but it does it by looping through /opt/stack/new right?18:15
sdagueso it doesn't matter what devstack does if the repo is there in d-g18:16
*** emagana_ has quit IRC18:16
*** _nadya_ has quit IRC18:16
*** hashar has joined #openstack-infra18:16
*** emagana has joined #openstack-infra18:16
mordredsdague: right - but why would the repo be in /opt/stack/new ?18:16
*** HeOS has joined #openstack-infra18:17
mordredsdague: let me go look at code real quick18:17
*** gokrokve has quit IRC18:17
sdaguebecause d-g puts it there18:17
sdaguedevstack doesn't clone in the gate18:17
*** emagana has quit IRC18:17
mordredsdague: there are _several_ stackforge projects that are in projects.txt that subscribe to changes and tie themselves to what's in g-r18:19
*** linuxmohan has quit IRC18:19
*** MarkAtwood has quit IRC18:19
*** emagana has joined #openstack-infra18:19
sdaguemordred: yeh, and I think they can all break us this way18:19
mordredsdague: I disagree - and if they can, I think it's just a bug to fix18:19
sdaguethey just haven't yet18:19
mordredsdague: ok. it's like this. the problem is triggered by "setup_package_with_req_sync"18:20
mordredsdague: that is called when we call setup_install on a directory18:20
mordredwe do not do that except for directories that we install from git as part of what we're doing18:20
mordredsince ironic uses dib, the way we had it installing was via setup_install from its git repo18:21
mordredwhich meant we were reqs syncing it18:21
mordredif we install from pip, we will not req sync it18:21
mordredthe other stackforge projects are not mentioned or installed from git as part of a normal intergrated run18:21
mordredso they will not trigger it18:21
*** funzo has quit IRC18:21
sdague....18:21
mordredessentially, we are installing a tool using integrated gate mechanisms without it being part of teh integrated gate18:22
mordredwe need to install it not using integrated gate mechanisms18:22
sdagueok18:23
sdagueso your patch actually as to propose to devstack icehouse then, correct?18:24
*** doude has joined #openstack-infra18:24
mordredyah.18:24
*** arxcruz has quit IRC18:24
mordredalthough I think we should investigate the other suggestion SpamapS was making18:24
mordredof being able to enable via  flag installing from git18:24
mordredbecause that seems useful for devs18:24
EmilienMmordred: hey, I'm a potential mentor for OPW and I'm trying to clarify https://wiki.openstack.org/wiki/OutreachProgramForWomen/Ideas#Community - hogepodge told me there ls some potential work in Puppet to do for openstack-infra. First question, do you have a blueprint or something listing what need to be done? (i.e. ask.o.o, etc)18:25
sdagueyeh... so that's probably part and parcel with the library proposal I have on the list18:25
mordredsdague: yup18:25
fungiEmilienM: we have a huge low-hanging-fruit one actually!18:25
mordredsdague: apparently tripleo have written the logic in shell already18:25
bookwarare there any docs/talks/slides explaining jenkins-job-builder parameters substitution? I mean at which point {smth} is substituted and what are the entry points and priorities for parameter values: project definition only, project + global defaults section, or project + global + job definition? I am trying to go through the sources but maybe there is a shortcut..18:26
fungiEmilienM: it's getting discussed at the infra team meeting in half an hour too18:26
EmilienMfungi: the program takes 3 months afik18:26
*** xyang0 has quit IRC18:26
fungiEmilienM: yeah, this can be chunked up as needed i think. nibalizer is currently getting stuff lined up for it18:26
EmilienMfungi: any pointer?18:26
fungiEmilienM: and then we need volunteers for refactoring chunks18:26
*** mmedvede has quit IRC18:26
SpamapSmordred: we just have "packages" or "source" for element/install.d .. it may not be consumable for this exact case.18:27
* SpamapS goes on errand18:27
EmilienMfungi: what I need first is to identify a user story, like they have in other projects (i.e. https://wiki.openstack.org/wiki/OutreachProgramForWomen/Ideas#Glance_-_Swift_ranged_uploads)18:27
*** IAX has quit IRC18:27
*** funzo has joined #openstack-infra18:27
*** IAX has joined #openstack-infra18:28
*** jp_at_hp has quit IRC18:28
*** ivar-lazzaro has joined #openstack-infra18:28
fungiEmilienM: http://specs.openstack.org/openstack-infra/infra-specs/specs/puppet-modules.html is the one which comes to mind18:28
clarkbbookwar: http://ci.openstack.org/jenkins-job-builder/definition.html something like that?18:28
EmilienMfungi: this is awesome.18:28
*** ramashri has joined #openstack-infra18:28
EmilienMfungi: can I take it as example of an OPW internship?18:29
mordredEmilienM: absolutely18:29
*** emagana has quit IRC18:29
EmilienMmordred: cool, thx18:29
sdagueSpamapS: direct git url might be handy, as there are a lot of tripleo repos, and I'm not familiar with them18:30
*** hashar has quit IRC18:30
*** _buzz has quit IRC18:30
fungiEmilienM: though of course we have lots of other things which would also be good possibilities for opw work, i think. doesn't have to be puppet-specific18:30
openstackgerritA change was merged to openstack-infra/project-config: Fix indentation in rally-designate job config  https://review.openstack.org/12434618:31
EmilienMfungi: feel free to give me specs links18:31
fungiEmilienM: well, we have http://specs.openstack.org/openstack-infra/infra-specs/ approved and also https://review.openstack.org/#/q/status:open+project:openstack-infra/infra-specs,n,z proposed for consideration18:33
nibalizerEmilienM: I'd be happy to help with that18:34
EmilienMgreat!18:34
fungiEmilienM: but there's tons more which we could probably mine out of our bug list and turn into low-level specs as well, for better clarity18:34
*** hashar has joined #openstack-infra18:35
clarkbsdague: fungi mordred any opposition to having the ironic sideways grenade job as a nonvoting check job on ironic, devstack, d-g, and grenade?18:36
sdaguenope18:36
EmilienMfungi: I try to focus the internship on Puppet18:36
openstackgerritAndreas Jaeger proposed a change to openstack-infra/project-config: Group stackforge projects in layout.yaml  https://review.openstack.org/12458918:36
openstackgerritAndreas Jaeger proposed a change to openstack-infra/project-config: Add more checks for zuul/layout.yaml  https://review.openstack.org/12459318:36
openstackgerritAndreas Jaeger proposed a change to openstack-infra/project-config: Use tox -e zuul for layout test  https://review.openstack.org/12510118:36
*** kgavini has quit IRC18:36
*** kgavini_ has joined #openstack-infra18:36
SpamapSsdague: https://git.openstack.org/cgit/openstack/diskimage-builder/tree/elements/base/extra-data.d/99-enable-install-types18:37
fungiclarkb: no opposition from me18:37
clarkbkk /me approves this change to make that a non voting job18:37
openstackgerritA change was merged to openstack-infra/project-config: Stop using deprecated TESTR_FULL option  https://review.openstack.org/12440418:37
nibalizerEmilienM: yes, pulling out the modules, then maybe applying Modulesync to them, maybe writing tests, there is lots to do18:37
SpamapSsdague: of course, we have things split into individual executables, rather than bash functions, so as I say, it might not help18:37
EmilienMnibalizer: I like this one18:38
openstackgerritA change was merged to openstack-infra/infra-specs: HP owns everything. I own nothing.  https://review.openstack.org/12018818:38
*** tonytan_brb has joined #openstack-infra18:38
*** tonytan4ever has quit IRC18:38
* SpamapS does need to disappear for a bit18:38
*** david-lyle_afk has quit IRC18:38
AJaeger_clarkb, fungi, anteaya: Could you review my patch series soonish (starting at https://review.openstack.org/125101), please? I want to avoid more merge conflicts...18:38
clarkbAJaeger_: sure18:39
*** cdent has quit IRC18:39
clarkbAJaeger_: these should be self testing right?18:40
openstackgerritA change was merged to openstack-infra/project-config: Move notifications for pycadf from oslo to keystone  https://review.openstack.org/12444918:40
AJaeger_clarkb: the last patch - 124593   - is the test18:40
AJaeger_clarkb: so far it only tests the ordering of stackforge projects. Let's do one step at a time ;918:41
*** thedodd has joined #openstack-infra18:41
anteayanibalizer: ha ha ha18:41
anteayaAJaeger_: looking18:41
openstackgerritA change was merged to openstack-infra/project-config: Promote sideways ironic grenade tests to non-voting checks  https://review.openstack.org/12456218:42
*** IAX has quit IRC18:42
AJaeger_thanks, clarkb and anteaya18:42
nibalizeranteaya: are you laughing because i implied we might write tests?18:42
*** mmedvede has joined #openstack-infra18:42
AJaeger_thanks, fungi as well18:42
*** AzherKhan has quit IRC18:42
*** khaazher has joined #openstack-infra18:43
*** khaazher has quit IRC18:43
*** khaazher has joined #openstack-infra18:43
*** khaazher has quit IRC18:43
anteayanibalizer: no that you admit you own nothing18:43
anteayanibalizer: your patch just merged in specs18:44
anteayaha ha ha18:44
fungianteaya: nibalizer: yeah, it seemed pointless to wait for a second +2 on that patch18:44
anteayanibalizer: I love that you write test, might write tests or imply that you might write tests18:44
anteayanibalizer: keep doing that18:44
openstackgerritA change was merged to openstack-infra/project-config: Move python33 test for Keystone to experimental  https://review.openstack.org/12456318:45
*** VijayTripathi has joined #openstack-infra18:45
*** e0ne has quit IRC18:45
anteayaclarkb: I'll wait for you on 12510118:45
openstackgerritMatthew Treinish proposed a change to openstack-infra/subunit2sql: Add manifest file to include base alembic config  https://review.openstack.org/12515118:45
*** yolanda has joined #openstack-infra18:46
openstackgerritSteve Martinelli proposed a change to openstack-infra/config: Remove projects_alphabetized from tox.ini  https://review.openstack.org/12260018:46
*** VijayTripathi has quit IRC18:46
openstackgerritJeremy Stanley proposed a change to openstack-infra/project-config: Add more checks for zuul/layout.yaml  https://review.openstack.org/12459318:46
*** jgrimm is now known as zz_jgrimm18:46
fungiAJaeger_: corrected your commit message18:47
AJaeger_thanks, fungi18:47
*** MarkAtwood has joined #openstack-infra18:47
sdagueSpamapS: yeh, this is probably different enough that it's not going to directly translate18:47
openstackgerritSteve Martinelli proposed a change to openstack-infra/config: Remove jjb from tox.ini  https://review.openstack.org/12515218:47
*** openstackgerrit has quit IRC18:47
*** openstackgerrit has joined #openstack-infra18:48
openstackgerritA change was merged to openstack-infra/project-config: macros: Manage metadata.json file in puppet-lint jobs  https://review.openstack.org/12456918:49
*** berendt has quit IRC18:49
clarkbAJaeger_: fungi: it looks like the merge template check is disabled18:51
clarkbAJaeger_: fungi: should it be enabled? The commit message seems to imply that it is18:51
AJaeger_clarkb: it's disabled because it fails18:51
AJaeger_we need to merge first https://review.openstack.org/12458818:51
AJaeger_sorry, I had a few too many loose ends coming together ;(18:52
fungiAJaeger_: don't apologize. great work18:52
*** hdd has quit IRC18:53
AJaeger_clarkb: I suggest to move the patch series in and I enhance 124588 to enable the merge check there18:53
arosen-home1Hi, i started seeing this weird nova failure in my CI but i think it's related to configuration: http://paste.openstack.org/show/117243/ . One thing that looks odd is nova complains saying: Nova requires libvirt version 0.9.11 or greater.18:53
arosen-home1If i'm using devstack-gate and nodepool with a ubuntu14.04 image i figured i'd already have that no?18:54
*** zz_jgrimm is now known as jgrimm18:54
*** unicell has quit IRC18:56
krotscheckSo, I want to contribute code that I wrote under the apache license to pecan, however pecan is licensed under BSD. What’s the appropriate approach there?18:56
clarkbAJaeger_: sounds good18:57
krotschecki.e. There’s a part of StoryBoard that would be awesome for anyone using Pecan.18:57
fungikrotscheck: they're compatible licenses, but pecan effectively becomes redistributed under the intersection of those licenses18:57
clarkbor if you own the copyright on it then give it to pecan as bsd18:57
krotscheckHrm.18:58
clarkb(assuming you are ok with those license terms)18:58
fungikrotscheck: unless you can convince the copyright holder (probably your employer?) to relicense18:58
anteayaclarkb: so do you want to +A those in the right order?18:58
krotscheckWell, _I_ don’t care.18:58
fungiyeah, what clarkb sais18:58
clarkbanteaya: yup18:58
fungisaid18:58
krotscheckBut HP probably does.18:58
anteayaclarkb: thanks18:58
AJaeger_thanks, clarkb !18:58
* krotscheck wanders off to see who to ask about that.18:58
anteayanice job AJaeger_18:58
clarkbkrotscheck: lifeless18:58
fungier, union of those licenses, which leaves the intersection of their allowances i guess18:58
fungimeetin' time, peoplez18:59
pleia2oh boy, time flies19:00
*** VijayTripathi has joined #openstack-infra19:00
clarkbso soon19:00
*** bcourt has quit IRC19:01
*** andreykurilin_ has joined #openstack-infra19:02
*** hashar has quit IRC19:02
*** zaphod42_ has quit IRC19:02
anteayaclarkb: yeah I think it will be a bit before lifeless is online much, so perhaps someone else might be good to find19:02
*** rushiagr is now known as rushiagr_away19:02
*** zaphod42 has joined #openstack-infra19:02
*** chuckC has quit IRC19:03
*** krtaylor has joined #openstack-infra19:03
*** ChuckC_ is now known as ChuckC19:03
clarkbhe is usually around this time of day19:03
*** chuckC_ has joined #openstack-infra19:03
*** praneshp has quit IRC19:04
*** gokrokve has joined #openstack-infra19:04
*** annegentle has quit IRC19:04
*** hashar has joined #openstack-infra19:04
*** mmedvede has quit IRC19:05
*** otherwiseguy has joined #openstack-infra19:05
*** praneshp has joined #openstack-infra19:06
openstackgerritA change was merged to openstack-infra/project-config: Use tox -e zuul for layout test  https://review.openstack.org/12510119:06
*** mrmartin has quit IRC19:07
*** ramashri has quit IRC19:07
*** tonytan_brb has quit IRC19:08
*** praneshp has quit IRC19:08
*** rfolco has quit IRC19:08
openstackgerritA change was merged to openstack-infra/project-config: Group stackforge projects in layout.yaml  https://review.openstack.org/12458919:09
*** jgrimm is now known as zz_jgrimm19:10
stevemarAJaeger, haha, sorry about the can of worms! haha19:10
AJaeger_stevemar: thanks a lot for fixing these!19:13
*** dane_leblanc has quit IRC19:13
openstackgerritSteve Martinelli proposed a change to openstack-infra/config: Remove projects_alphabetized from tox.ini  https://review.openstack.org/12260019:15
openstackgerritA change was merged to openstack-infra/project-config: Add more checks for zuul/layout.yaml  https://review.openstack.org/12459319:16
*** thedodd has quit IRC19:16
AJaeger_clarkb: stevemar's is a cleanup patch for the project-config split...19:16
*** lttrl has quit IRC19:17
*** bcourt has joined #openstack-infra19:17
*** mmedvede has joined #openstack-infra19:17
openstackgerritSteve Martinelli proposed a change to openstack-infra/config: Remove jjb from tox.ini  https://review.openstack.org/12515219:17
*** zz_jgrimm is now known as jgrimm19:18
stevemarAJaeger, why is there a tox env called 'upstream'19:18
stevemarthis doesn't seem to do anything...19:18
lifelessclarkb: ?19:19
*** dane_leblanc has joined #openstack-infra19:19
openstackgerritSteve Martinelli proposed a change to openstack-infra/config: Remove irc testenv from tox.ini  https://review.openstack.org/12516419:19
clarkblifeless: krotscheck had questions about licensing some storyboard apache2 licensed code as bsd udner pecan19:19
krotschecklifeless: See hipchat19:20
krotscheckOr we can talk here.19:20
krotschecklifeless: This piece in particular -> http://git.openstack.org/cgit/openstack-infra/storyboard/tree/storyboard/api/middleware/cors_middleware.py19:21
*** ildikov has joined #openstack-infra19:21
fungistevemar: i thought that one was intended to test whether specified "upstream" git repos for new projects were cloneable19:22
lifelesswhere is pecans upstream ?19:23
openstackgerritSteve Martinelli proposed a change to openstack-infra/config: Remove undefined upstream environment from tox.ini  https://review.openstack.org/12516719:23
stevemarfungi, my reasoning here ^19:23
lifelesskrotscheck: I'm not signed into hipchat atm19:23
lifelesskrotscheck: I can if needed19:23
krotschecklifeless: I’ll paste.19:23
stevemarfungi, upstream and projects_alphabetized where combined into 'projects' some time ago19:23
lifelesskrotscheck: that middleware looks eerily similar to r1chardj0n3s_afk's middleware19:23
fungistevemar: oh, yep19:23
stevemari don't know how that ever worked19:24
lifelessexcept that yours is raw WSGI19:24
krotschecklifeless: Oh?19:24
stevemarfungi, but they are no longer needed, so let's kill it with fire19:24
lifelesskrotscheck: https://review.openstack.org/#/c/120964/19:24
fungistevemar: agreed19:24
openstackgerritAndreas Jaeger proposed a change to openstack-infra/project-config: Add merge-check to pylockfile, test for merge-check  https://review.openstack.org/12458819:24
krotscheckOh!19:24
krotscheckNEat!19:24
*** baoli has quit IRC19:25
*** melwitt has joined #openstack-infra19:25
AJaeger_clarkb: 124588 enables the merge-check19:25
*** shashankhegde has quit IRC19:26
lifelesskrotscheck: so to answer the question, contributing a patch or minor enhancement to bsd projects is I believe covered by the whitelist on 'does not need OSRB approval'19:26
*** david-lyle_afk has joined #openstack-infra19:26
*** rfolco has joined #openstack-infra19:26
*** kgavini_ has quit IRC19:26
*** david-lyle_afk is now known as david-lyle19:26
lifelesskrotscheck: I'd need to sign into the VPN to check the exact thing - but have a look on the OSRB web site you should find the flow chart pretty easily19:26
stevemarsorry for all the patches AJaeger :\19:27
openstackgerritA change was merged to openstack-infra/storyboard-webclient: Added draft build  https://review.openstack.org/12493419:27
krotschecklifeless: Excellent. I’ll go look at that.19:27
lifelesskrotscheck: if BSD isn't whitelisted then an OSRB proposal would be needed, but it would be trivial and actioned without issue19:27
lifelesskrotscheck: (also we should propose whitelisting BSD if its not)19:27
dougwiganteaya: do you know if the brocade CI is supposed to be up right now?  (i think it should).  logs are 404.   https://review.openstack.org/#/c/123468/19:27
krotschecklifeless: I agree. Once the meeting’s over I have to switch over to the vpn anyway, so I’ll do that.19:28
AJaeger_stevemar: one big patch would have worked as well ;) But great to have it cleaned up!19:28
lifelesskrotscheck: that said, r1chardj0n3s_afk 's one looks like it will do what you need, and we can avoid having two of them L)19:28
lifelesss/L/:19:28
anteayadougwig: I don't, can you check third-party-announce archives and tell me what the lastest says?19:28
stevemarAJaeger, clean is good!19:28
*** [HeOS] has joined #openstack-infra19:29
krotschecklifeless: Correct. Though I do feel that it’s generic enough to be useful for pecan, not just pecan + oslo.19:29
lifelesssure19:30
*** baoli has joined #openstack-infra19:30
lifelessso is r1chardj0n3s_afk 's, modulo the use of oslo.config19:30
*** HeOS has quit IRC19:30
krotschecklifeless: Though I think r1chardj0n3s_afk implementation is probably more thorough than mine.19:30
lifelessanyhow, I don't have a dog in the fight19:30
dougwiganteaya: looks like you disabled one of their ADX CI on 9/4 for the same reason.  this is their vyatta CI.19:30
dougwignothing else in the archive.19:31
anteayakk thanks for looking19:31
anteayaI appreciate it19:31
*** jergerber has joined #openstack-infra19:32
openstackgerritAntoine Musso proposed a change to stackforge/python-jenkins: add get_version method  https://review.openstack.org/12488519:32
anteayaclarkb: can you disable 10692 brocade-oss-service Brocade Vyatta CI DL-GRP-VYATTA-OSS@Brocade.com19:33
*** dprince has quit IRC19:34
*** tonytan4ever has joined #openstack-infra19:35
*** sdake_ has quit IRC19:35
*** _nadya_ has joined #openstack-infra19:38
fungianteaya: i can get it19:38
anteayafungi: thanks19:39
fungianteaya: disabled 10692 just now19:39
anteayathank you19:39
anteayawill compose an email to -announce after infra meeting19:39
openstackgerritK Jonathan Harker proposed a change to openstack-infra/project-config: Clean up bashate failures  https://review.openstack.org/12486219:40
openstackgerritK Jonathan Harker proposed a change to openstack-infra/config: Clean up bashate failures  https://review.openstack.org/11894419:41
*** ramashri has joined #openstack-infra19:41
openstackgerritAndreas Jaeger proposed a change to openstack-infra/project-config: Remove github-org from projects.yaml  https://review.openstack.org/12516919:42
*** praneshp has joined #openstack-infra19:42
*** atiwari has quit IRC19:46
krotscheckIn the interest of landing more storyboard things, I have this! https://review.openstack.org/#/c/124936/19:46
bookwarclarkb: I was curious about how to pass variables into macros definition. i mean whether variables defined for a project are global or not. It seems they aren't. In this example http://fpaste.org/137904/21043561/ , without lines 13-14 it doesn't work.19:48
*** cdent has joined #openstack-infra19:48
clarkbbookwar: ya they are not global19:49
*** e0ne has joined #openstack-infra19:50
*** zehicle_at_dell has joined #openstack-infra19:51
*** Sukhdev has quit IRC19:51
*** annegent_ has quit IRC19:51
openstackgerritSteven Dake proposed a change to openstack-infra/project-config: Add Kolla Project  https://review.openstack.org/12445319:51
anteaya319:53
*** shashankhegde has joined #openstack-infra19:53
*** radez is now known as radez_g0n319:55
*** zns has quit IRC19:56
*** thedodd has joined #openstack-infra19:58
*** changbl has quit IRC19:58
*** zns has joined #openstack-infra19:58
*** changbl has joined #openstack-infra19:59
clarkbasselin: git clone https://git.openstack.org/openstack-infra/project-config then make changes?20:00
*** rlandy has quit IRC20:00
clarkbso I have bad news on the dib front. we appear to have completely broken all image updates20:00
sdaguefungi: do you have a review up for pulling all the havana jobs?20:00
clarkbno images have been updated in a week :(20:00
*** sarob has joined #openstack-infra20:00
jesusaurusclarkb: so maintain a fork?20:00
clarkbjesusaurus: I mean thats what you are wanting to do...20:01
*** baoli has quit IRC20:01
asselinclarkb, yeah, I'm thinking I'll mirror that repo locally, and create a branch with my stuff, and periodically merge in master.20:01
clarkbyou are literally saying "I want your stuff + my changes" (a fork)20:01
AJaeger_clarkb, oops ;( is there no way to get notified about such failures?20:01
clarkbAJaeger_: not today. I am trying to look into it, but other eyes would be helpful20:01
clarkbmordred: fungi ^20:01
asselinclarkb, ok 'fork'.20:01
krtayloradalbas, ^^^20:02
*** mfer has quit IRC20:02
pleia2hogepodge: might be interested in the tail end of our meeting today, talked about 3rd party ci: http://eavesdrop.openstack.org/meetings/infra/2014/infra.2014-09-30-19.00.html20:02
*** annegentle has joined #openstack-infra20:02
openstackgerritA change was merged to openstack-infra/elastic-recheck: add links to specific bugs  https://review.openstack.org/12492120:03
*** AJaeger_ has quit IRC20:04
fungisdague: i think it already merged?20:04
fungijust a sec20:04
sdagueI just had grenade-icehouse job run on tempest20:04
openstackgerritA change was merged to openstack-infra/elastic-recheck: fix Launchpad link name  https://review.openstack.org/12492820:04
*** sarob has quit IRC20:04
*** baoli has joined #openstack-infra20:05
*** JayJ has quit IRC20:06
asselinanteaya, krtaylor I won't be around for next monday's 3rd party meeting, but I think clarkb's answer makes sense. This is my plan: 'fork'. More specifically: " I'll mirror that repo locally, and create a branch with my stuff, and periodically merge in master."20:06
*** JayJ has joined #openstack-infra20:06
*** IAX has joined #openstack-infra20:07
anteayaasselin: thanks for letting us know your status and plans20:07
anteayaasselin: let us know how that workflow works for you20:07
anteayaasselin: and if you have any suggestions for how to improve it20:07
anteayaas well as documenting it20:07
asselinok, sure. probaly won't get to it until Oct 15th though. Yes, if there's a recommended place to document it, let me know.20:08
krtaylorasselin, understood, we can bring it up Monday20:08
asselinthanks. going to lunch20:09
krtayloranteaya, related, how about we discuss any possible third-party test system impact of system-config rename in Monday's third-party meeting?20:09
*** kgiusti has left #openstack-infra20:10
*** mfer has joined #openstack-infra20:10
*** marun has quit IRC20:10
*** cnesa15 has quit IRC20:10
clarkbjesusaurus: asselin: in particular we wanted to separate the system configs from the project configs so that others doing ci similar to us can reconsume the system but have different project configs20:12
anteayakrtaylor: sure, I may or may not know20:13
anteayakrtaylor: since I don't run a third party ci20:13
jesusaurusclarkb: yeah, it just wasnt clear to me if the intent was for project-config to be forked, mimicked, or thrown out the window by downstreams20:13
anteayakrtaylor: but I will share what I know, if I know anything20:13
*** stevemar has quit IRC20:13
*** ociuhandu has quit IRC20:13
*** shashankhegde has quit IRC20:13
anteayajesusaurus: I think the format should be used but no necessarily the values20:14
anteayas/no/not20:14
jesusaurusbut which of those is best is probably dependent on the downstream and what they are trying to do20:14
clarkbfungi: mordred http://paste.openstack.org/show/117261/ finally found the exception. I am working on a fix20:14
clarkbjesusaurus: yup agreed20:14
krtayloranteaya, well, I would expect that there wouldn't be as much impact for those systems that have already integrated or forked project-config20:15
krtayloranteaya, mainly a heads-up I guess20:15
krtayloranteaya, I'll add it to the agenda20:15
anteayakrtaylor: can't hurt20:16
fungiclarkb: thanks. i'm in far too many conversations at once to pay attention to what that was about just now20:16
fungioh, scrollback says image updates went stale. ick20:16
openstackgerritClark Boylan proposed a change to openstack-infra/nodepool: Properly reference nodepool config in image update  https://review.openstack.org/12518620:16
clarkbfungi: mordred ^ that should fix image updates20:16
sdaguefungi: can you see if the ER bot is lost?20:17
sdagueit should work now that ES is back20:17
*** IAX has quit IRC20:17
*** kgavini has joined #openstack-infra20:18
fungisdague: the er bot was dead since august when someone changed how its challen list is configured. did that get fixed yet?20:19
fungis/challen/channel/20:19
sdaguefungi: oh... hmph20:19
*** Guest43089 has joined #openstack-infra20:20
sdagueif you pastebin an error, I can fix post TC meeting20:20
*** annegent_ has joined #openstack-infra20:20
fungisdague: k, i can do that post meeting20:20
*** Guest43089 is now known as annegentle_20:20
*** annegent_ has quit IRC20:20
*** annegentle has quit IRC20:21
*** homeless has quit IRC20:21
*** mordred is now known as jeblair20:21
*** jeblair is now known as mordred20:21
*** homeless has joined #openstack-infra20:22
clarkbmordred: reviewing 125186 when you are done meeting would be great. also chown: invalid user: `jenkins:jenkins' is happening in my precise builds now. checking trusty real quick tpoo20:23
mordredclarkb: sounds like jenkins user is not being added20:24
clarkbmordred: ya, not sure why yet20:24
arosen-home1I'm trying to do a release of a python-client on pypi. I've added a tag 1.0.0a1 and pushed that to gerrit. Now, when i run: python setup.py sdist upload -r python-congressclient i get this: http://paste.openstack.org/show/117265/20:25
arosen-home1am i missing something more?20:25
*** _nadya_ has quit IRC20:25
mordredarosen-home1: we actually usually just have infra publish things to pypi for you when you push a tag20:25
*** marun has joined #openstack-infra20:25
arosen-home1mordred:  ah i have to add infra to my pypi package i registered i guess?20:26
mordredarosen-home1: I've never seen that error though20:26
mordredarosen-home1: openstackci is the account you want20:26
clarkbalso don't use setup.py for that stuff. iirc it does so via http not https. its super secure20:26
*** shashankhegde has joined #openstack-infra20:26
clarkbwe use that tool dstufft wrote to do it instead20:26
arosen-home1mordred:  i added openstackci as a owner role. Will the openstackci eventually retry to upload the package?20:27
*** koolhead17 has quit IRC20:27
*** ihrachyshka has quit IRC20:27
clarkbarosen-home1: no, you need to correct jobs in zuul and its a one shot thing20:27
*** julim has quit IRC20:27
clarkbif stuff fails we manually kick it20:27
fungiclarkb: arosen-home1: "that tool dstufft wrote" is twine20:28
clarkbty I forgot the name20:28
*** kgavini__ has joined #openstack-infra20:28
*** kgavini__ has quit IRC20:29
arosen-home1is there a place i can look to see if this worked or failed?20:30
arosen-home1I guess if it failed I have to bug you guys?20:30
*** kgavini has quit IRC20:30
fungiarosen-home1: yeah, we have logs20:30
arosen-home1where would i see the logs for such a thing?20:31
clarkbarosen-home1: I don't think you have the jobs20:31
fungiarosen-home1: logs.openstack.org will have a log for the git sha of the tyag itself (you can find it by running git show-ref tagnum)20:31
clarkbhttps://git.openstack.org/cgit/openstack-infra/project-config/tree/zuul/layout.yaml#n4732 doesn't list pypy jobs20:31
clarkband ya what fungi said http://logs.openstack.org/firsttwoofthatsha1/sha120:31
fungiit'll appear in a pre-release or release subdirectory there20:32
*** zns has quit IRC20:32
*** cdent has quit IRC20:33
openstackgerritAaron Rosen proposed a change to openstack-infra/project-config: Add publish-to-pypi job for python-congressclient  https://review.openstack.org/12518920:33
arosen-home1so once ^ merges will the pypi package be pushed ?20:33
arosen-home1or i guess the tag has to be done after this is setup.20:34
fungiarosen-home1: no, but a subsequent tag would cause the next version to be20:34
arosen-home1k, thanks again guys.20:35
*** datsun180b has quit IRC20:35
*** juzuluag has quit IRC20:36
*** stevemar has joined #openstack-infra20:37
*** koolhead17 has joined #openstack-infra20:37
rfolcoclarkb, my forked openstack_functions.py has params['OFFLINE_NODE_WHEN_COMPLETE'] = '1', jenkins says the slaves are offline, but nodepool says the slaves are ready... so the job finishes and slaves are not deleted by nodepool. I can't find any evidences in the logs... could you give some advice to point me to the right direction ?20:39
*** andreaf has quit IRC20:39
*** mmedvede has quit IRC20:39
*** boden has quit IRC20:39
*** sacharya has left #openstack-infra20:39
*** andreaf has joined #openstack-infra20:39
clarkbrfolco: are you publishing jenkins zmq events to nodepool? you need those so that nodepool can know when to delete the nodes20:39
*** andreaf has quit IRC20:40
clarkbyou also need them for nodepool to know a node is used so I think that explains it if it is missing20:40
*** andreaf has joined #openstack-infra20:40
rfolcoclarkb, this one?20:40
rfolcoJenkins Event Publisher (via ZMQ PUB SUB)20:40
rfolcoThis plugin publishes Jenkins build events in JSON as a ZMQ Publisher.20:40
*** datsun180b has joined #openstack-infra20:40
stevemarfungi, err, question about test hooks for dsvm20:40
stevemardo post_test_hook functions get run *after* devstack is torn down?20:41
clarkbrfolco: ya20:42
rfolcoclarkb, I'll make sure I have 0.0.3 and re-rest20:42
*** hashar has quit IRC20:42
clarkbrfolco: nodepool needs that feed to know when to update the nodes20:42
*** bhuvan has quit IRC20:42
*** zns has joined #openstack-infra20:43
fungistevemar: not sure. i'd need to read through the devstack-gate scripts (unless you beat me to it)20:43
rfolcoclarkb, perhaps some configs set at http://git.openstack.org/cgit/openstack-infra/config/tree/modules/openstack_project/manifests/jenkins.pp do not exist in my fork :( -- thanks for the pointers.20:43
*** hashar has joined #openstack-infra20:43
stevemarfungi, hmmmm on it now... https://github.com/anteaya/project-config/blob/4cbe40aa954b7ac6bcd12b98039bfc1ae6e3247e/modules/openstack_project/files/jenkins_job_builder/config/devstack-gate.yaml20:44
stevemartheres a pre, post, and just gate_hook20:44
*** ppai has left #openstack-infra20:45
*** rmcall has quit IRC20:47
*** sarob_away is now known as sarob20:48
*** akerr has joined #openstack-infra20:48
jogoclarkb: when would tempest consume say nova unit tests?20:48
openstackgerritKhai Do proposed a change to stackforge/python-jenkins: generate a code coverage html report  https://review.openstack.org/12519420:49
clarkbjogo: if/when nova unittests expose code to test nova in a particular way20:49
EmilienMfungi: clarkb: when you got 2 min, could you have a look at the new project accepted by TC ? 'tripleo-puppet-elements into OpenStack'  https://review.openstack.org/#/c/12456020:49
clarkbjogo: in exactly the same way hacking does it20:49
clarkbjogo: test code is consumable code.20:49
*** yamahata has joined #openstack-infra20:49
jogoclarkb: I don't follow20:49
jogowhat is the hacking example?20:49
clarkbjogo: hacking consumes pep8's unittests20:49
clarkbor is it flake8? one of them20:50
jogohacking is a flake8 plugin20:50
clarkbyes but hacking has tests20:50
clarkbthat specifically import another projects tests20:50
clarkbto make use of them20:50
jogoclarkb: it has a non-voting job that does that yes20:50
clarkbjogo: it was voting last I saw20:50
jogoclarkb: we must be thinking of different things20:51
jogoclarkb: the case I envision is: I cleanup some nova unit tests. My patch only touches files in nova/tests/*20:51
jogowhat does running a dsvm job do for that case?20:51
*** bhuvan has joined #openstack-infra20:52
clarkbjogo: it makes sure that your changes to importable and consumable code do not break consumers of that code20:52
jogoclarkb: nothing in dsvm should be consuming that code20:53
akerrHi infra.  If anyone familiar with zuul pipeline layouts could take a look at my yaml it'd be really helpful.  We're deploying a gerrit/zuul/jenkins/nodepool code review/gate system internally to vet our cinder driver code before we push out to the community and I can't seem to get this pipeline to trigger properly.  I think it might have something to do with my "require" tags: http://paste.openstack.org/show/117267/   The idea is that this should20:53
akerr trigger when someone Release+1 comments, if all the other required votes are present20:53
clarkbjogo: maybe not today20:53
jogotoday we don't explicitly state that, but we should state that IMHO20:53
clarkbjogo: I disagree20:54
jogonova/tests is NOT a stable API20:54
jogoand consume at your own risk20:54
clarkbjogo: then it should not be in the nova namespace20:54
clarkbjogo: it is packaged and included whenever you install nova20:54
jogoclarkb: not all of nova namespace is a stable API20:54
clarkbwhich is intentional20:54
*** oanufriev has quit IRC20:54
*** dimtruck is now known as zz_dimtruck20:55
jogoclarkb: we have a lot of code in nova namespace that isn't considered a stable API20:55
clarkbjogo: yes and its the massive problem that nova development is currently facing20:55
clarkbyou dont' have stable apis so everything must be in tree going through one group of people20:55
jogoclarkb: that is hardly a related issue20:56
jogoyes they are both about APIs20:56
fungisdague: here's the traceback from the recheck bot, when you have a moment to go over it http://paste.openstack.org/show/117268/ (though the traceback i saw a few weeks ago was a parsing issue with the channels list in the yaml file, so this may be yet new bitrot on top)20:56
jogoclarkb: we will never have a case where we say all the things are stable APIs20:56
jogoin nova the only thing that is a true public API is the REST API toeay20:56
clarkbjogo: thats fine. I am just trying to point otu that we explicitly do unittests this way for the reasons that hacking has for it20:56
jogotoday20:56
clarkbjogo: and to compeltely shutdown that usecase is a change in status quo20:57
*** oanufriev has joined #openstack-infra20:57
jogoclarkb: I still don't see what hacking has to do with this20:57
clarkbjogo: hacking consumes other projects unittests20:57
jogoclarkb: its not a use case we should even allow20:57
clarkbwhy not?20:57
clarkbhacking does it20:57
jogoclarkb: hacking just needs the code to be well python20:57
clarkband mordred has specifically fougth for this use case20:57
clarkbunitests are useful code20:57
clarkbmake them consumeable20:58
jogoclarkb:  sure unit tests are useful code20:58
jogoclarkb: but they are not a stable API20:58
jogowe change them at will20:58
clarkbjogo: the __init__.py has been pretty stable20:58
clarkbjogo: and honestly has to be out of necessity so that the 10k tests in nova continue to work20:58
clarkbor is it test.py in nova?20:59
jogojust because something hasn't changed much doesn't mean its a public stable API20:59
mordredjogo: more things need stable public apis20:59
mordredthat we keep saying that we dont' have stable apis is lazy20:59
clarkbjogo: no but in this case it is because it must be stable20:59
clarkbotherwise you have 10k flaky tests20:59
mordredand it's time to do less of it, not more20:59
jogomordred: but unit tests?20:59
mordredeverything20:59
*** mfer has quit IRC20:59
mordredEVERYTHIN20:59
mordredpeople are building giant systems on top of this21:00
mordredfor us to try to say we don't expect people to use our internal apis is bananas21:00
jogomordred: if everything is a stable API we cannot change anything21:00
mordredsure we can21:00
mordredwe just do it sensibly21:00
clarkbmordred: going to bug yo uabout https://review.openstack.org/#/c/125186/1 again :)21:00
jogomordred: we don't say that, we say this is not a public API it may change. but go for it21:00
*** hdd has joined #openstack-infra21:00
mordredjogo: we actively do much worse than that21:01
*** sarob is now known as sarob_away21:01
krotscheckCan a core take a look at this projects-config patch? It’s a minor tweak - https://review.openstack.org/#/c/124936/21:01
jogomordred: sure21:01
mordredjogo: we regularly change interface definitions and the unittests that use them21:01
*** baoli has quit IRC21:01
mordredjogo: this broke ironic several times this cycle21:01
jogomordred: all I am saying is that nova unit tests should not be deemed a public (and this stable) API21:01
mordredand it's just pure laziness21:01
jogowe can move around code at will in unit tests21:01
mordredjogo: I agree, actually, I don't think unit tests need to be a public api21:01
mordredBUT21:01
*** MarkAtwood has quit IRC21:02
mordredI do believe that making sure that a change to the unittets don't break the ability to consume the unittest SYSTEM is important21:02
clarkbkrotscheck: I would but I already have :)21:02
jogomordred: well ironic was using an interface that was *not* public21:02
krotscheckclarkb: Clone yourself!21:02
mordredjogo: that should embarrass a lot of people21:02
clarkbkrotscheck: if only21:02
jogomordred: sure21:02
mordredsince that's a CLEAR plugin point21:02
jogomordred: we sadly have plugins to unstable APIs21:02
jogowhy? I don't know21:03
mordredthat should stop. right. now.21:03
mordredit's very easy to stop it21:03
mordredwe just stop it21:03
mordredit's code21:03
mordredwe can make choices21:03
mordredlike not having unstable plugin apis21:03
mordredwhatever it is now? stable21:03
mordreddone21:03
mordrednow be careful to only add things, etc.21:03
jogomordred: I have to double check the entry points  etc. that we have21:03
jogobut we have a lot21:03
*** r1chardj0n3s_afk is now known as r1chardj0n3s21:03
mordredyes. we do. we became production software a long time ago21:04
mordredthose should all be considered defacto API contracts21:04
r1chardj0n3skrotscheck: are you around? I just woke up and saw your discussion about my CORS middleware with lifeless :)21:04
sdagueall those things can be changed in a backwards compatible way, you just need to get into the mindset of it21:04
jogomordred: so going back to the nova unit test and running tempest example21:04
krotscheckr1chardj0n3s: Nope. I am a turing machine.21:04
r1chardj0n3skrotscheck: cool21:04
* krotscheck actually means yes.21:04
r1chardj0n3skrotscheck: I don't discriminate :)21:04
mordredjogo: the only reason I'd personally fight to keep that21:04
fungikrotscheck: i have this halting problem i'd like your help with ;)21:05
krotscheckfungi: oh?21:05
clarkbfungi++21:05
mordredjogo: is to catch places where someone does something stupid in a file that causes nova itself to not be runnable for $reason21:05
fungikrotscheck: http://en.wikipedia.org/wiki/Halting_problem21:05
mordredjogo: I'm fairly certain unittests themselves would catch that21:05
krotscheckfungi: How do you feel about having this halting problem i’d like your help with [/eliza]21:05
mordredjogo: however, I'm not sure we've got a great way to express that in zuul21:05
fungiheh21:05
openstackgerritA change was merged to openstack-infra/project-config: Call build:draft instead of build for js-draft build  https://review.openstack.org/12493621:05
mordredjogo: nor do I really believe it'll gain us a ton21:05
mordredjogo: to warrant the development effort21:06
clarkbjogo: mordred ya the benefits are minimal21:06
r1chardj0n3skrotscheck: just happy to answer questions if you have then. there is also the related spec for oslo kilo https://review.openstack.org/#/c/119485/21:06
clarkbpeople don't even run pep8 before pusing code21:06
clarkbso not running the harde rto run integration tests doesn't optimize anything21:06
krotscheckr1chardj0n3s: Looking.21:06
*** zz_dimtruck is now known as dimtruck21:07
jogomordred: so your saying you want to keep the integration jobs around for unit test only changes for the case where breaking a unit test actually breaks nova21:07
r1chardj0n3skrotscheck: no idea whether it's even useful for your use-case as I don't know what that is ;)21:07
mordredjogo: yah. since we don't actually test nova's unittests with tempest, I doubt running them on those changes will catch many other things21:07
sdaguejogo: so zuul doesn't support the kind of change logic you are proposing21:07
krotscheckr1chardj0n3s: My use case is CORS support on the api at storyboard.openstack.org, ergo yes, it’s useful.21:07
mordredand since we use discovery to look for tests, import errors should get caught21:08
sdagueyou can't not run tests because of the output of diff21:08
r1chardj0n3skrotscheck: ok, cool21:08
krotscheckr1chardj0n3s: And using oslo is definitely an option, though I might suggest that CORS is something that pecan can benefit from too :)21:08
mordredjogo: but my main concern is that it's not an easy fix21:08
*** otherwiseguy has quit IRC21:08
mordredjogo: because of what sdague is saying21:08
sdagueI've actually spoken with jeblair on that some21:08
jogosdague: how much work wound it be to support something like that somewhere (maybe outside of zuul?)21:08
r1chardj0n3skrotscheck: pecan?21:08
mordredjogo: it has to be in zuul21:08
sdaguejogo: you can't outside of zuul21:08
krotscheckr1chardj0n3s: Touche21:08
jogosdague mordred: so that is a fair point, that it isn't a quick change21:08
krotscheckr1chardj0n3s: Because middleware intercepts before pecan.21:09
*** otherwiseguy has joined #openstack-infra21:09
* krotscheck will ponder this.21:09
* r1chardj0n3s is ignorant of pecan21:09
sdaguejogo: it actually hits one of my hot button issues of people throwing around "we should just change this part of our process"21:09
sdaguewithout understanding the tooling implication21:09
clarkbsdague++21:09
*** mmedvede has joined #openstack-infra21:09
krotscheckr1chardj0n3s: is a path->object mapper for wsgi21:09
mordredkrotscheck, r1chardj0n3s: dhellmann may have thoughts21:09
clarkbsdague: we are very good at saying "X should just Y"21:09
r1chardj0n3skrotscheck: ok, thanks21:09
HenryGclarkb: Do you have a minute? I have a question about modules/openstack_project/manifests/slave_db.pp21:09
sdagueand not realizing that process without supporting tooling is just giant frustration, pain, and disillusionment21:10
fungianyone who wants to make sweeping proposals is welcome to throw together a rough draft patch to our tooling to explain how they expect it should work21:10
clarkbmordred: trusty seems broken too21:10
krotscheckdhellmann: Thoughts on CORS Middleware as a part of Oslo, Pecan, and/or other things?21:10
*** boden has joined #openstack-infra21:10
jogoI am uncomfortable with the logical extreme of this argument. That we shouldn't do X because our tooling today doesn't support it21:10
*** boden has quit IRC21:10
ryanpetrellofrom a maintainer of pecan standpoint, I'd prefer it *not* be in pecan21:10
clarkbmordred: for dib building so might be related to config split. I am debugging21:10
mordredclarkb: AWESOME21:10
r1chardj0n3skrotscheck, dhellmann: I would prefer that CORS be unencumbered21:11
mordredjogo: yah. agree21:11
ryanpetrelloI tend to shy away from implementing things in pecan's core if I can help it; I try to keep the feature set as sparse as possible21:11
clarkbjogo: thats not the argument21:11
anteayaHenryG: best to just ask the question21:11
*** boden has joined #openstack-infra21:11
clarkbjogo: the argument is we haev a problem where we say lets just do X ifnra21:11
clarkbjogo: and the expectation is it will just happen tomorrow21:11
clarkbjogo: see libvirt testing discussion21:11
r1chardj0n3skrotscheck:, dhellmann: that is, it shouldn't require the deployer to have to worry about what pecan might be to use it21:11
*** boden has quit IRC21:11
*** aysyanne has quit IRC21:11
dhellmannkrotscheck: putting CORS in pecan would make it only available to the projects using pecan, which isn't a lot of them yet21:12
HenryGanteaya: Yeah, I should know better by now :(21:12
jogoclarkb: this is exactly why I am bringing this up here to see what the issues/implications etc are21:12
krotscheckr1chardj0n3s, dhellmann: Ditto with Oslo.21:12
stevemarfungi, ughhh, i don't think a lot of projects are doing functional tests properly :(21:12
fungior more to the point, some suggestions which can be described simply at a high level very well may entail extremely complex implementations21:12
anteayaHenryG: :)21:12
*** boden has joined #openstack-infra21:12
marunsdague: then zuul is broken and needs to change21:12
mordredstevemar: I'm certain of that21:12
mordredmarun: aroo?21:12
marunre: we can't limit the jobs run based on the diff21:12
ryanpetrellodhellmann: krotscheck: is there a way to accomplish the CORS from a strictly WSGI middleware approach (without something pecan-specific)21:12
dhellmannkrotscheck, r1chardj0n3s, ryanpetrello : middleware seems like the right approach here21:12
ryanpetrellothis would benefit the entire community21:12
jogobroken is a strong word21:12
*** boden has quit IRC21:12
dhellmannryanpetrello: https://review.openstack.org/#/c/119485/6/specs/kilo/cors-middware.rst21:12
marunjogo: broken -> file a bug -> fix it21:12
fungistevemar: perhaps. i thought many of them were doing it with an overridden gate_hook?21:12
*** boden has joined #openstack-infra21:13
jogomarun: you should file a zuul bug then :)21:13
krotscheckdhellmann, r1chardj0n3s: I agree that middleware’s the right approach. I’m questioning the oslo.config dependency.21:13
mordredmarun: I believe "doesn't have that feature at the moment because we quite literally have not ever defined that as being a use case we want" might be more what you meant21:13
*** krtaylor has quit IRC21:13
stevemarfungi, some are, some aren't - i think only oslo is doing it correctly21:13
mordredit's not a bug21:13
mordredit's not a bug21:13
mordredit's not a bug21:13
*** boden has quit IRC21:13
mordredit's a very conscious choice21:13
jogomordred: why?21:13
HenryGHow do I apply modules/openstack_project/manifests/single_use_slave.pp to machines for the check-neutron-dsvm-functional job?21:13
dhellmannkrotscheck: ah, ok, I misunderstood the question21:13
mordredbecause it's not a thing we've ever wanted to do21:13
jogomordred: what was the conscious choice21:13
marunmordred: I think it's interesting to hear you say that it isn't a use case that has been considered21:13
r1chardj0n3skrotscheck: it has to be configured *somewhere* and it makes sense to me to not add a new configuration file just for CORS21:13
mordredand in fact21:13
jogomordred: why?21:13
mordredmost of the time21:13
mordredwe want to NOT do it21:13
*** boden has joined #openstack-infra21:13
marunmordred: It's a huge waste of time for everyone21:13
mordredbecause doing it is an optimization21:13
*** sarob_away is now known as sarob21:13
dhellmannkrotscheck: maybe we should talk about that on the spec review?21:14
stevemarfungi, a lot of projects are kicking off a project specific post_gate_hook, but it's running tests, which don't make sense... since devstack has been torn down21:14
clarkbakerr: sorry too much stuff going on right now. have you compared it to the upstream pipeline?21:14
r1chardj0n3sdhellmann, krotscheck: that makes sense, yes21:14
mordredand so far, in the project, we have tended to solve optimizations not by running less tests21:14
mordredbut by increasing our ability to run tests21:14
mordredso21:14
krotscheckr1chardj0n3s, dhellmann: Indeed. Off to gerrit!21:14
mordredin general21:14
*** boden has quit IRC21:14
* krotscheck rides off into the code review system.21:14
mordreda feature to limit what tests we run as a way to scale the system21:14
marunmordred: ...and we're pretty screwed as a resuult21:14
mordredis a completely different desgin21:14
mordredand one to not be taken lightly21:14
HenryGDarn. I mean, How do I apply modules/openstack_project/manifests/slave_db.pp to machines for the check-neutron-dsvm-functional job?21:14
clarkbhow are we screwed?21:14
mordredmarun: no21:14
jogomordred: aren't we at the scale where its time to do optimizations21:14
marunmordred: yes21:14
*** boden has joined #openstack-infra21:14
mordredwe are screwed because people are writing shitty code21:14
fungistevemar: some of them want devstack to configure things but not leave all the services running, i think21:14
marunmordred: pushing patches sucks so damn hard21:14
mordredand the code IS actually broken21:14
akerrclarkb: let me compare them, thanks for the response21:15
marunmordred: having merge queues that are so damn long21:15
mordredso people should stop wanking with features and actually fix the damned bugs21:15
clarkbakerr: we do very similar in the upstream gate pipeline so it would be a good one to compare to21:15
marunmordred: patch velocity only due to a lack of reviewer resources?21:15
*** boden has quit IRC21:15
anteayaHenryG: are you asking in the context of your third party ci environment?21:15
marunmordred: No, as much a result of the impossible task of trying to ensure overall openstack quality on .every.single.project.merge.proposal21:15
*** boden has joined #openstack-infra21:15
fungipatch velocity limited due to developers playing with pet features rather than fixing what's already broken in their projects21:15
marunfungi: sure21:16
*** boden has quit IRC21:16
fungiand also due to integrating projects which were not stable enough to actually be plugged into one another21:16
mordredmarun: and you think quality woudl be better by increasing the number of less well tested patches we can land?21:16
HenryGanteaya: No. This is for the check-neutron-dsvm-functional community job.21:16
marunfungi: and that is as much due to openstack not requiring good testing21:16
*** mjturek has quit IRC21:16
mordredmarun: or just that it would be easier for individual devs to land a patch?21:16
marunfungi: pretending we can bolt it on with tempest and the gate21:16
fungiabsolutely21:16
*** boden has joined #openstack-infra21:16
mordred++21:16
anteayaHenryG: do you have a patch in place that I can look at?21:16
marunmordred: I want to see dramatically increased requirements around in-tree project testing21:16
mordredmarun: I do too21:16
anteayaHenryG: or in process?21:16
*** boden has quit IRC21:17
openstackgerritA change was merged to openstack-infra/subunit2sql: Add manifest file to include base alembic config  https://review.openstack.org/12515121:17
fungii think it was probably tractable to test it that way a while back, but we scaled past it being effective more quickly than anyone had an opportunity to implement more focused testing solutions21:17
HenryGanteaya: Not yet, I am trying to figure out how. Not very good at puppet so not sure where to start.21:17
mordredmarun: I'm pushing back angrily because it would be a LOT of work to implement a "only test this patch when X changes" feature, and I'm pretty sure the amount of gain we'd get is nothing21:17
*** boden has joined #openstack-infra21:17
marunmordred: I want it to be harder to merge patches that aren't tested.   But I also hope that we can all recognize the futility of running so many integrated jobs in the hopes that it ensures quality.21:17
marunmordred: It doesn't.21:17
mordredcompared to the potential for rampant abuse21:17
marunmordred: It increases developer friction with no real benefit21:17
anteayaHenryG: let's work on me understanding what you are doing, what are you trying to accomplish?21:17
*** boden has quit IRC21:17
clarkbmarun: so of that statement I am not so sure21:17
anteayaHenryG: keep in mind everyone else is engaged in another topic and I'm slow21:18
mordredmarun: I disagree with "no real benefit"21:18
marunclarkb: we are as likely to surface a race condition after a merge as before21:18
*** boden has joined #openstack-infra21:18
clarkbmarun: if we take neutron for example. Quality there is much better than it was for many reasons, including the extra integration testing we do against it21:18
jogolets say each component of our system works 99% of the time. If we have 10 pieces to the system we get a success rate for the collective system of 90%21:18
marunclarkb: really?21:18
clarkbmarun: we run double the tests and thankfully the whole suite now21:18
HenryGanteaya: Sure, the bug is https://bugs.launchpad.net/neutron/+bug/137298121:18
clarkbmarun: yes neutron is double tempested21:18
uvirtbotLaunchpad bug 1372981 in neutron "Neutron functional tests can't target both postgres and mysql in the same test run" [Medium,Confirmed]21:18
clarkbmarun: and it appears to have helped21:18
marunclarkb: and that sucks balls21:18
clarkbsignificantly21:18
*** boden has quit IRC21:18
*** bhuvan has quit IRC21:18
anteayaHenryG: thanks, that is a start21:18
clarkbbecause in the last 6 months neutron has done what it has been trying to do for 2.5-3 years21:19
marunclarkb: we now have twice as many chances to surface unrelated race conditions that block us from merging21:19
*** boden has joined #openstack-infra21:19
anteayayay!!21:19
mordredmarun: so what?21:19
mordredmarun: so don't merge21:19
* anteaya throws a party at what clarkb just said21:19
fungimarun: recall that not too long ago there was basically no integration testing of neutron at all. just a stub because nothing worked. the amount of testing it's under now compared to them is tremendous21:19
mordredso let's all stop wroking on new features and new vendor plugins21:19
marunclarkb: I have a pretty good chance of surfacing that kind of condition with a tox.ini change that has zero functional implications21:19
marunthis is progress?21:19
*** boden has quit IRC21:19
mordredand let's do nothing but work on race bugs21:19
marunmordred: fsck the race bugs21:19
fungier, compared to then21:19
marunthat's the symptom21:19
HenryGanteaya: One way to do it would be to patch devstack so it can install both DB backends.21:20
*** boden has joined #openstack-infra21:20
jogomordred: if you can make that happen awesome21:20
marunlet's write in-tree functional and integration tests that validate functionality for that service alone21:20
mordredmarun: yes. let's21:20
openstackgerritSteve Martinelli proposed a change to openstack-infra/project-config: Change OSC functional tests to gate_hook  https://review.openstack.org/12520221:20
mordredPLEASE21:20
*** boden has quit IRC21:20
mordredmarun: but let's not turn off what little we have until we've actually done that21:20
marunlet's cover off exceptional cases _before_ we try to add all the complexity of other services21:20
HenryGanteaya: But the unit-test jobs install both backends via the puppet script I referred to above.21:20
mordredbecause what we have may suck21:20
stevemarfungi, ^21:20
mordredbut it's FAR better than nothing21:20
marunmordred: let's recognize the tradeoff involved too, though21:20
HenryGanteaya: So I would like to try the same for the functional job.21:21
*** boden has joined #openstack-infra21:21
jogomordred: you should try writing a tempest patch21:21
marunmordred: MOOR TESTS BETTER is broken21:21
anteayaHenryG: it looks like from the bug report you would be best served by spending some time with mtreinish in -qa21:21
marunlet's recognize that and find a good balance21:21
mordredmarun: totally21:21
mtreinishanteaya: what did I do?21:21
*** boden has quit IRC21:21
mordredmarun: all I'm saying is that we should not remove what we have until we have something new and better21:21
anteayamtreinish: you are just awesome21:21
anteayamtreinish: keep doing that21:21
*** boden has joined #openstack-infra21:21
mordredmarun: because our current experience is that less tests == immediate breakage fail pain21:21
marunmordred: I'm saying we should take a hard look at the cost/benefit of what we have, and adjust21:21
stevemarfungi, thank you sirrrr21:22
HenryGmtreinish: It may be hard to read the backlog here :D21:22
mordredand that has been consistent every time something has failed positive over the last 2 years21:22
*** boden has quit IRC21:22
marunmordred: if the cost of running a given job has a potential payoff of catching a race/regression, great.21:22
marunmordred: run it21:22
*** IAX has joined #openstack-infra21:22
*** akerr has left #openstack-infra21:22
*** boden has joined #openstack-infra21:22
mtreinishanteaya: cool, wfm :)21:22
marunmordred: if there is a low chance that job has value - do not run it.21:22
mtreinishHenryG: heh yeah that's why I asked?21:22
mordredmarun: this happens21:22
marunmordred: save the resources, and more importantly the developer time21:22
mordredmarun: consistently21:22
marunmordred: I think we've conflated 2 things that should be independent21:23
*** boden has quit IRC21:23
marunmordred: One is keeping a given project tree stable21:23
fungireplace the current poorly-targeted tests with more focused testing? good. drop the current tests in hopes that better testing emerges before everything falls apart? _bad_21:23
marunmordred: The other is keeping openstack as a whole stable21:23
marunThese are different things21:23
*** boden has joined #openstack-infra21:23
mordredmarun: right now we do neither21:23
mordredwell21:23
*** boden has quit IRC21:24
marunmordred: I'm saying, we need to consider these goals separately if we have any hope of achieving them21:24
mordredsure21:24
marunwe're doing both, badly, right now21:24
jogomarun: ++21:24
sdaguemarun: so help21:24
*** boden has joined #openstack-infra21:24
anteayaboden check your client21:24
*** bhuvan has joined #openstack-infra21:24
marunsdague: that's always your refraing,21:24
mordredthis is why I agree that we need functional tests21:24
marunsdague: refrain21:24
mordredmarun: but why I believe that we need them FIRST21:24
marunsdague: Do you honestly think I'm not doing all I can to move this forward?21:24
*** boden has quit IRC21:24
marunsdague: I pushed for how many months to get agreement that we needed to move in a different direction21:25
marun?21:25
clarkbmordred: how do I break inside of a dib build?21:25
*** boden has joined #openstack-infra21:25
mordredclarkb: no clue. SpamapS ^^ ?21:25
marunsdague: I hope you can trust at some point that I am working on this.21:25
mordredlifeless: ^^ ?21:25
*** homeless has quit IRC21:25
mordredmarun: I know you're working on it - but please trust in the opposite direction too21:25
*** boden has quit IRC21:25
mordredmarun: and know that when you start with "zuul is broken" the response might not be too immediately welcome21:26
clarkbanteaya: can your propose the cahnge you suggested in 124254? ianw is out this week21:26
*** homeless has joined #openstack-infra21:26
*** boden has joined #openstack-infra21:26
anteayaclarkb: /me looks21:26
marunmordred: it's qualified21:26
*** boden has quit IRC21:26
marunmordred: if zuul cannot dynamically determine the jobs to run based on a given patch, then a bug exists that needs to be fixed21:26
*** boden has joined #openstack-infra21:27
marunmordred: I think you know as well as I that saying something dramatic is sometimes done for the attention.21:27
mordred:)21:27
mordredwhat21:27
mordredI never do that21:27
*** tonytan4ever has quit IRC21:27
marunhah21:27
anteayaclarkb: yep, going to go for a walk first then will propose21:27
clarkbanteaya: ty21:27
*** boden has quit IRC21:27
mordredmarun: thing is - as I mentioned above - that's a conscious design decision21:27
sdaguemarun: so the impact of that statement is basically people that do try to help with these things, like myself, decide it's completely not worth it because there is so much stone throwing from the outside21:28
*** boden has joined #openstack-infra21:28
mordredmarun: and although I do think that the conversation about whether or not it's a useful design or whether we need a new one is a good discussion21:28
sdagueso if that's your intent, you win21:28
*** boden has quit IRC21:28
marunOk21:28
marunLet's air some dirty laundry then21:28
*** bcourt has quit IRC21:28
*** _buzz has joined #openstack-infra21:28
marunand sdague, you can call it 'stone-throwing' all you want, but I think honesty is important21:28
*** boden has joined #openstack-infra21:28
clarkbmordred: fungi: for some reason pupept isn't finding openstack_project::single_use_slave in the dib builds, but it should be in the modulepath. so next step for me is to set up a breakpoint and figure it out21:29
*** boden has quit IRC21:29
marunI think infra is overloaded to the point that developer productivity doesn't really rate in the priority list21:29
mordredclarkb: oh wow. that's amazing21:29
mordredmarun: I disagree very strongly21:29
mordreddeveloper producitivy is super important21:29
*** boden has joined #openstack-infra21:29
marunmordred: how much code do you push?21:29
*** doude has quit IRC21:29
mordredmarun: a GIANT amount21:29
fungii think it can be implemented if we really want it. but i also question the effective gain... how many changes touch only non-code? and are those even changes worthy of prioritizing if the code itself is so broken it's painful to test?21:29
marunmordred: how many hours do you waste waiting on gate queues?21:30
marunmordred: on rechecks?21:30
marunmordred: Do you think this is acceptable?21:30
*** boden has quit IRC21:30
mordredmarun: I don't wait on gate queues - I tend to have at least 100 different tasks in my queue at any point in time21:30
marunmordred: Or that contributes to developer productivity?21:30
clarkbmordred: fungi actually I think its a bug in the build image script. I think I can sort this21:30
fungisame here. i parallelize my work and pick up whatever's ready for a next step as opportunity allows21:30
*** boden has joined #openstack-infra21:30
marunmordred: So you've optimized your workflow to account for how broken our infrastructure is21:30
marunmordred: good for you21:30
*** dhellmann is now known as dhellmann_21:30
mordredmarun: wow21:30
mordredno21:30
marunmordred: not everyone has that luxury21:30
mordredour infrastructure is NOT broken21:30
lifelessclarkb: mordred: Read the manual21:30
marunmordred: that's a statement of opinion21:31
mordredit does exactly what we ask it to very well21:31
clarkblifeless: :P I figured that was coming21:31
mordredmarun: openstack is broken21:31
*** boden has quit IRC21:31
mordredour infrastructure shows that quite regularly21:31
marunmordred: it is broken from the perspective of developer productivity21:31
lifelessclarkb: search for debug in README21:31
marunmordred: we've optimized to try to keep openstack stable21:31
mordredthen I disagree with what you find productive21:31
*** boden has joined #openstack-infra21:31
mordredbecause "landing patches" is not necessarily productive21:31
fungiour infrastructure will actually run jobs which are sanely written extremely well. the projects the code is coming from are full of bugs we're successfully identifying21:31
mordred"fixing openstack" is21:31
marunmordred: to the detriment of developer productivity21:31
mordrednope21:31
clarkblifeless: ty21:31
mordrednot productive21:31
marunmordred: there is no open discussion of the tradeoffs involved21:31
lifelessclarkb: or break should hit it too :)21:31
*** boden has quit IRC21:32
mordredyou're assuming that mean time to land patch should be minimized21:32
marunmordred: so zuul is DESIGNED to avoid dynamic discovery of the jobs to run21:32
mordredI disagree21:32
*** VijayTripathi has quit IRC21:32
*** boden has joined #openstack-infra21:32
*** e0ne has quit IRC21:32
mordredmarun: zuul is designed to solve the problems we actually have, which is keeping up with the aggregate project velocity21:32
marunmordred: and that indicates to me - and nothing you're saying is contradicting this view - that developer productivity is not considered important21:32
mordredwhich is MASSIVE21:32
fungiif developers on some projects would rather stick their collective heads in the sand and pretend it's okay to be completely broken, then i question the reason for even testing those projects (or including them in openstack)21:32
mordredmarun: I still disagree with yoru definition of productive21:32
*** boden has quit IRC21:32
krotscheckRandom fun thing: When clicking on gate-storyboard-webclient-js-draft in gerrit you now get a fully functional client. (caveat - small bug in login that I’m fixing)21:32
mordredmarun: when the project in question is not broken, things run well21:33
marunmordred: do you honestly think that your opinion is canonical here?21:33
krotscheckExample: https://review.openstack.org/#/c/104004/21:33
mordredit's not my job to fix all the problem21:33
mordredmarun: no, I do not21:33
*** boden has joined #openstack-infra21:33
marunmordred: that I'm a lone voice, and that I'm uninformed?21:33
marunI get that we need to converge on stability21:33
mordredmarun: I think taht you are representing a well known viewpoint that it's currently hard for developers to land a patch21:33
*** boden has quit IRC21:33
marunand that we are hamstrung in the absence of effective project-specific functional testing21:33
mordredI agree with that21:33
* krotscheck pokes himself to speed things up a bit....21:34
*** bcourt has joined #openstack-infra21:34
*** boden has joined #openstack-infra21:34
mordredwe need much more of that and we need it to be much better21:34
fungialso, zuul does make dynamic determination on what jobs to run and when. very dynamic decisions. it just doesn't have the necessary dynamism right now to vault your "fix all teh typoz" change through without running unit tests21:34
marunBut nowhere do I hear any suggestion that we can tune the amount of integration testing to the benefit derived21:34
*** e0ne has joined #openstack-infra21:34
mordredbut you just described a failure in the projects, not in the automation of them21:34
*** boden has quit IRC21:34
mordredit is completely in the power of the projects to add good functional tests21:34
marunour current infra is an accident21:34
*** boden has joined #openstack-infra21:35
mordredno. it is most certainly not21:35
mordredand it is completely in our power to consistenlty run those tests21:35
marunmore than 2 years ago everyone realized that openstack was broken21:35
jogoat the current scale of the integrated gate, I don't think we can ever get to the point where dsvm-tempest works 100% of the time21:35
*** boden has quit IRC21:35
marungating began, first on devstack exercises21:35
mordrednope21:35
marunrather than fixing the problem - the projects were poorly tested21:35
mordredthat is factually inaccrate21:35
marunwe tried to bolt on quality21:35
marunthat much is _not_ factually inaccurate21:35
*** boden has joined #openstack-infra21:35
mordredthe project started with gating from literally commit one21:35
marunwe've tried to bolt on quality for 2+ years21:36
mordredand has always had it21:36
marunand it doesn't work21:36
clarkbanteaya: note 124254 needs an update too, I will push that as soon as I can test it21:36
marunwe get that now21:36
mordredso we've been gating on tests for 4 years21:36
*** boden has quit IRC21:36
marunmordred: I play fast and loose with my dates because I've forgotten how long I've been here I guess.21:36
*** boden has joined #openstack-infra21:36
mordredmarun: you are describing the advent of devstack21:36
mordredbefore that, openstack was uninstallable21:37
lifelessstill is:P21:37
marunlifeless++21:37
fungizing!21:37
*** boden has quit IRC21:37
mordredand the devs would occasionally say "look, here is some puppet"21:37
mordredbut it wouldn't work21:37
lifelessstill do :P21:37
* lifeless isn't being helpful21:37
* lifeless goes21:37
*** boden has joined #openstack-infra21:37
*** boden has quit IRC21:38
*** e0ne has quit IRC21:38
*** boden has joined #openstack-infra21:38
mordredmarun: listen, we're in the weeds - I VERY MUCH agree and support the idea of much better targetted functional testing21:39
*** boden has quit IRC21:39
mordredmarun: I think we'll die without it21:39
mordredmarun: and we probably should have had it quite a while ago21:39
*** boden has joined #openstack-infra21:39
*** andreaf has quit IRC21:39
clarkbanteaya: so lets hold off on that second change for now21:39
*** boden has quit IRC21:39
mwhudsonoh yes21:40
*** andreaf has joined #openstack-infra21:40
fungi(and for that matter, most projects could use some unit tests which are actually unit tests)21:40
*** boden has joined #openstack-infra21:40
mwhudsonso last week i came in here and asked "how do i run tempest like gate does"21:40
mwhudsonand apparently this week, the answer is different? :)21:40
clarkbmwhudson: no?21:40
*** boden has quit IRC21:40
marunmordred: I think we agree on a lot of this stuff.21:40
mwhudsonclarkb: something about moving to dib?21:40
*** boden has joined #openstack-infra21:41
clarkbmwhudson: yes, and we have been planning the move for at least half a year. But it isn't quite there yet and the old way continues to work21:41
fungimwhudson: our providers finally started running openstack! (well, started providing glance)21:41
marunmordred: I think there are 2 areas we can improve in advance of our future functional test utopia.21:41
*** boden has quit IRC21:41
marunmordred: The first one is to allow dynamic determination of jobs to run.21:42
*** boden has joined #openstack-infra21:42
mwhudsonclarkb: ah ok21:42
fungimwhudson: the current mechanism nodepool uses was, as much as anything, invented to work around teh fact that we couldn't upload our own server images into cloud providers21:42
*** boden has quit IRC21:42
marunmordred: The cost of running unnecessary integration jobs (i.e. on test-only changes) is not justified.21:42
mwhudsonfungi: ah21:42
*** boden has joined #openstack-infra21:42
*** andreykurilin_ has quit IRC21:42
mordredmarun: I understand and respect that position. I disagree with your premise that dynamic determination of jobs to run will help things for developers21:43
fungimwhudson: now that we can, that opens us up to be able to build an image locally and then push it to all providers/regions, so massive performance gain21:43
*** eharney has quit IRC21:43
mordredmarun: I actually believe that it will make their experience worse21:43
marunmordred: Why?21:43
mwhudsonfungi: are you building such images yet?21:43
mordredmarun: beacuse we do not have the functional tests yet21:43
*** boden has quit IRC21:43
*** pcm_ has quit IRC21:43
fungimwhudson: and will also make it possible for other parties (devs, third-party testing operators, et cetera) to consume/reuse the images we're building nightly21:43
marunmordred: Hold on, we're not talking about the same thing.21:43
mordredmarun: so my confidence that targetted running will not lead to an increase in race bugs is low21:43
*** boden has joined #openstack-infra21:43
mwhudsonfungi: right21:43
marunmordred: This has zero to do with race bugs.21:43
mordredmarun: ah! excellent! I love it when that happens21:43
mordredmarun: when you say targetted running, what do you mean?21:44
*** boden has quit IRC21:44
fungimwhudson: we are building images now, yes, and have nodes which can boot from them and run experimental jobs successfully21:44
mwhudsonfungi: well ultimately i care about 3rd party testing on arm64 so i guess i'll have to make my own images21:44
mwhudsonbut that's ok21:44
marunmordred: If I am making a change to the test tree (verifiably non-functional), the integration jobs should not run.21:44
*** boden has joined #openstack-infra21:44
*** hashar has quit IRC21:44
fungimwhudson: right, there may still be gains there as well, though we haven't really pondered how to do cross-architecture nodepool... however if you have an arm-based nodepool server21:44
clarkbmarun: how do you verify that though?21:45
clarkbmarun: today the answer is your run the tests21:45
*** boden has quit IRC21:45
mwhudsoni'd also like to just run the tests myself on intel and see no failures, because i haven't managed that yet21:45
clarkbmarun: asserting in zuul config that it won't isn't quite the same21:45
marunclarkb: look at the patch, if the changes are only to neutron/tests, don't run integration jobs21:45
*** boden has joined #openstack-infra21:45
mordredmarun: right. but the generalized feature you're describing is "run tests less"21:45
marunclarkb: it has to be dynamic.  I know this isn't supported yet.21:45
clarkbmarun: so my argument earlier with jogo is that that is completely insufficient21:45
*** markmcclain has quit IRC21:45
marunmordred: no21:45
marunmordred: it's being smart about what we run21:45
clarkbat least with the assumptions we have today about our tests21:46
mordredmarun: in order to smartly run less tets21:46
marunmordred: you're hearing 'run tests less' betrays your bias towards 'all test runs have value'21:46
clarkbthey are part of consumable code21:46
*** boden has quit IRC21:46
marunand they dont21:46
mordredmarun: no, I do _not_ have that bias21:46
clarkband it is completely fair game for nova to import its tests or tempest to import nova's tests21:46
marunclarkb: Uh, what?21:46
*** boden has joined #openstack-infra21:46
fungimarun: so... you say this isn't about race conditions... yet... why do you care if the infra systems run a job guaranteed to pass because you're not modifying files affecting it?21:46
clarkbmarun: its python... you can do this. there are reasons to do this21:46
marunclarkb: Why in _hell_ would that be allowable?21:46
clarkbmarun: because there may be useful code in one project's tests that another wants to consume21:46
*** boden has quit IRC21:47
marunclarkb: the code in question that does that s*** should break.21:47
clarkbsee hacking21:47
lifelessironic needed to do that for instance21:47
mordredmarun: maybe you're ceilometer and you want to check that doing your thing doesn't cause nova to regress21:47
lifelesswith the nova driver21:47
*** boden has joined #openstack-infra21:47
mordredmarun: maybe you're ironic and need to check that your nova driver didn't break the existing nova unittests21:47
*** bswartz has quit IRC21:47
marunmordred: ffs21:47
marunthat's not a public api21:47
mordredmarun: and that's a lazy developer sentiment21:47
marunif someone is using something that isn't for public consumption, fsck them!21:47
mordredlike the plugin interface?21:47
lifelessso the public aspect has merit as an argument21:47
*** boden has quit IRC21:47
marunwe define api's for a reason21:48
mordredlike the plugin interface?21:48
fungigood thing most of our projects aren't actually for public consumption, as it turns out21:48
marunif someone uses something private - buyer beware21:48
lifelessbut right now the test suites are not annotated as private21:48
r1chardj0n3sthanks for the CORS spec review krotscheck, and your configuration point is valid. One way to solve the problem would be to require API authors to include the CORS middleware and configure it (apart from the allowed origins, of course). But that's a big mess...21:48
mordredlike the plugin interface?21:48
marunlifeless: so let's change that21:48
*** boden has joined #openstack-infra21:48
jogoclarkb: no it is not fair game for tempest to import nova's unit tests unless we agree on a stable API to dotaht21:48
*** dustins has quit IRC21:48
marunmordred: we need to stabilize a public api21:48
marunmordred: that's a battle we'll be fighting in neutron in kilo21:48
*** boden has quit IRC21:48
marunmordred: tests are arguably different, though21:48
mordredmarun: we need to admit that it's possible that anything in any of our projects has a contract in any way shape or form21:48
marunmordred: we get to decide that, anyway21:48
*** boden has joined #openstack-infra21:49
jogoclarkb: hacking is not an example that helps your point as hacking has *no* co-gating21:49
marunmordred: allowing something, because, that costs us, is the very definition of stupid21:49
krotscheckr1chardj0n3s: Is that really a big mess?21:49
clarkbjogo: but hackign does this with pep821:49
jogoclarkb: link?21:49
clarkbmordred: ^ did it not and didn't you yell at pep8 for breaking this?21:49
mordredok - two things21:49
r1chardj0n3skrotscheck: it would require getting *all* the projects to agree to having the middleware included, which I think could be troublesome :)21:49
*** boden has quit IRC21:49
mordredA) I do not believe unit tests are a public API21:49
mordredand I am not going to pretend that I do21:49
*** boden has joined #openstack-infra21:50
jogoclarkb: flake8 had some issues but they weren't hacking related and no co-gating was needed21:50
clarkbjogo: you can't cogate with them21:50
*** flaviof_zzz is now known as flaviof21:50
mordredB) I argue that unit tests should ship because sometimes someone MIGHT want to do what marun mentioned is a buyer beware activity and reuse them21:50
clarkbjogo: but you do consume their tests as a reconsumable thing21:50
*** boden has quit IRC21:50
mordredI do not think that we need to be concerned about that activity21:50
r1chardj0n3skrotscheck: having said that, it would be much nicer than requiring deployers to do it for every api21:50
marunmordred: in _production_ code?21:50
mordredother than allowing someone to make that decision if it makes sense for them21:50
jogoclarkb: oh that was something mordred did I think21:50
marunmordred: wth21:50
mordredmarun: I'm talking about test suites21:50
jogoclarkb: I have to double check if we still do that21:50
*** boden has joined #openstack-infra21:50
*** mriedem has quit IRC21:50
marunmordred: if a given project is going to pick things up, the responsibility is for them to find the problem21:51
mordredand devs who may want to do something with them in their own test suites21:51
mordredyes21:51
mordredI agree21:51
mordredyikes21:51
mordredclarkb is representing me as beieving something I do not beieve21:51
*** boden has quit IRC21:51
mordredbelieve21:51
clarkbmordred: sorry. However I think this is a perfectly valid use case21:51
marunclarkb: really21:51
marun?21:51
*** boden has joined #openstack-infra21:51
clarkbmordred: particularly since the tests must be internally consistent21:51
marunclarkb: why in heck?21:51
clarkbmarun: because of the hacking thing21:51
clarkband the ironic thing21:52
mordredI am now going to walk away from the laptop21:52
clarkbyour tests are reconsumable21:52
clarkbunless you say don't reconsume this in a pythonic way21:52
jogoclarkb: so the hacking thing. I think i know what it is now21:52
marunclarkb: that makes absolutely zero sense. I challenge you to find any developer on an openstack project that supports that position21:52
*** boden has quit IRC21:52
clarkbmarun: I think the ironic devs might21:52
marunclarkb: and once you've identified them, I'll personally tar and feather them!21:52
jogoclarkb: mordred wanted to use some code from pep8 that wasn't shipped since it was in there unit tests21:52
clarkbbecause nova has burned them so hard21:52
*** boden has joined #openstack-infra21:52
marunclarkb: isn't that there problem?  they do something dumb, they get burned.21:52
clarkbmarun: no, it isn't21:52
marunclarkb: changing the rules so they don't get burned, not a great tradeoff21:53
clarkbnova didn't say this is a private interface21:53
clarkbimport nova.tests21:53
clarkbits a thing21:53
*** boden has quit IRC21:53
clarkbthey are not marked private21:53
jogoclarkb: everything in nova is private unless we say so21:53
marunclarkb: it's common-fing sense, sorry21:53
clarkbif you want to assert this stance then do it properly21:53
*** r1chardj0n3s is now known as r1chardj0n3s_afk21:53
clarkbjogo: no no no no21:53
*** boden has joined #openstack-infra21:53
clarkbjogo: that is really bad python then21:53
jogoclarkb: private as in private API21:53
jogoas in not stable21:53
*** boden has quit IRC21:54
jogoclarkb: go to the nova room and ask there21:54
marunclarkb: on what planet does a project guarantee to clients that their _test suite_ is consumable and won't change in backwards-incompatible ways21:54
*** boden has joined #openstack-infra21:54
jogoor I can just get dansmith in here21:54
jogodansmith: ^21:54
clarkbjogo: marun I understand what you are saying..21:54
clarkbwhat I am saying is that you should be a bit more explicit about it if that is the stance21:54
*** zns has quit IRC21:54
marunclarkb: I consume tempest in the neutron poc, but I know they are going to break me at some point.21:54
*** boden has quit IRC21:54
clarkbfrom python's perspective you have done nothing to say don't do this21:54
dansmithum, what?21:55
marunpython lets you do anything21:55
*** boden has joined #openstack-infra21:55
jogodansmith: re: unit tests and stable whatnots21:55
*** bradjones has quit IRC21:55
jogoin backscroll21:55
*** sarob is now known as sarob_away21:55
fungipython also lets you identify private interfaces, and what isn't private is treated (by python) as public21:55
marunclarkb: we rely on convention and commonsense to limit what we can do to a sane subset of what's possible21:55
marunclarkb: it's the only way to work productively in a group21:55
*** boden has quit IRC21:55
lifelessI am quite sure the developer population consider the tests private21:55
lifelessbut as already said and agreed on they aren't annotated explicitly that way21:56
*** boden has joined #openstack-infra21:56
marun(because we're not using c++)21:56
clarkbmarun: you can move tests to the top level dir21:56
marunI mean, is the argument that the ironic guys were in the right by trying to reuse nova's test suite and not expecting to be broken?21:56
*** boden has quit IRC21:56
marunReally?21:56
dansmiththe question is whether or not nova.tests.* is a stable interface?21:56
fungiand there is some questionably legitimate use case when a project which is a plugin to another project wants to unit test the parent project with its code in place, that it may want to reuse those tests21:56
clarkbdansmith: no21:57
*** boden has joined #openstack-infra21:57
clarkbdansmith: the question is should they be reconsumable in a sane way21:57
marunAnd that it sets a precedent that we have to run the integrated jobs on every test change in case this happens again???21:57
clarkbdansmith: that may mean some bits are private others are stable21:57
clarkband not just nova21:57
*** boden has quit IRC21:57
dansmithclarkb: just to be clear, you mean someone other than nova importing nova.tests.something and using it?21:57
clarkbdansmith: yes, or nova itself within nova21:57
*** boden has joined #openstack-infra21:58
jogoclarkb: but if it doesn't have a '_' in the method name then it should be stable and public?21:58
dansmithwell, those are two totally different things, right?21:58
clarkbdansmith: not from the integration testing perspective21:58
*** boden has quit IRC21:58
dansmithI mean, nothing in nova imports anything from nova.tests, other than test code21:58
clarkbdansmith: if nova imports nova.tests you still want to test it in an integration scenario to make sure that public nova behavior isn't broken. Similar to something else importing it21:58
clarkbdansmith: sure, but it can...21:58
*** boden has joined #openstack-infra21:58
dansmithclarkb: it better not21:58
clarkbthere is nothing preventing it21:58
clarkbis what I am saying. we have presented the tests in a reconsumable format21:59
jogodansmith: we can add a hacking rule to enforce that21:59
dansmithjogo: yeah21:59
marunclarkb: and for that reason we need to run integrated jobs on every commit?21:59
* jogo does that21:59
*** boden has quit IRC21:59
*** jaypipes has quit IRC21:59
clarkbmarun: yes it is one way of catching it21:59
dansmithclarkb: I don't think I agree with the assertion that just because it's a module it's intended to be consumable21:59
marunclarkb: are you considering the cost of that?21:59
*** boden has joined #openstack-infra21:59
*** sarob_away is now known as sarob21:59
clarkbmarun: we do it today21:59
marunclarkb: and it's f'ing stupid21:59
clarkbmarun: so yes, and teh cost isn't that insane21:59
marunclarkb: for you maybe21:59
clarkbwe still merge a ton of code21:59
*** boden has quit IRC22:00
clarkbfungi: mordred I have sorted precise image builds22:00
marunclarkb: you are not feeling the pain, clearly22:00
fungii think it was just being used as an example of "you can't assume a change to unit test code won't impact other code sharing the same git repository, or other projects with which that software interacts"22:00
clarkbit is a bug in ianw's 124254 patch. pushing an update22:00
*** boden has joined #openstack-infra22:00
marunfungi: and it's a stupid example22:00
marunfungi: we avoid those problems by forbidding people from reusing the damn test suite22:00
marunfungi: a small price to pay for dramatically less gate cost / patch22:01
*** boden has quit IRC22:01
marunfungi: that's what I mean by tradeoffs22:01
clarkbmarun: then please forbid them from doing that22:01
marunassuming that something is good without considering the costs -> not sustainable22:01
marunclarkb: gladly!22:01
clarkbmarun: but the current stance has been to not do that22:01
marunclarkb: so let's agree to that22:01
clarkband sorry if I misunderstood mordred on that front22:01
*** boden has joined #openstack-infra22:01
dansmithclarkb: what do you want us to do? yell at people? or put some technical block in place?22:01
marunclarkb: I will make sure that nobody reuses the tests outside of the test tree22:01
clarkbdansmith: a technical block22:01
*** boden has quit IRC22:01
fungii would instead argue at a higher level that the perceived benefits from being able to avoid running integration test jobs for a project on changes to the fraction of its code which is not run-time functional is minimal22:01
clarkbdansmith: all you have to do is move tests to top level dir22:01
clarkbdansmith: but mordred is opposed to that22:02
marunclarkb: And we can ensure update zuul to support dynamic job discovery so we don't run integrated jobs unnecessarily.22:02
*** boden has joined #openstack-infra22:02
marunfungi: and you're wrong22:02
*** ociuhandu has joined #openstack-infra22:02
dansmithis there a real problem here, or are we just arguing?22:02
lifelessclarkb: no please don't suggeset top level tests dirs22:02
marunfungi: you're not trying to merge those patches22:02
*** changbl has quit IRC22:02
marunfungi: I am22:02
clarkblifeless: ok22:02
dansmithsorry if I missed it earlier in the scrollback22:02
lifelessclarkb: that messes with testr and global unique test ids22:02
*** boden has quit IRC22:02
marunfungi: The cost is real to me and everyone else trying to revise the massively terrible neutron test suite22:02
clarkblifeless: can you or mordred explain where not top level dirs is useful outside of reconsumable tests22:02
lifelessclarkb: we just went through the massive exercise to fix that most everwhere.22:02
lifelessclarkb: reporting on aggregated tests22:03
clarkblifeless: I thoguht we were fixing it for the hacking reasons22:03
*** boden has joined #openstack-infra22:03
fungiat best, it will encourage more/faster contributions from people writing in-tree (but not in-line) documentation, or writing unit tests22:03
marunfungi: good!22:03
lifelessclarkb: I don't know all the reasons. I know the reason I care about :)22:03
clarkblifeless: rgr22:03
marunfungi: easier/faster test contribution?22:03
marunfungi: where is the problem?22:03
marunfungi: we still have reviewer oversight.  the tests still run22:03
*** boden has quit IRC22:03
*** bhuvan has quit IRC22:03
*** weshay has quit IRC22:03
clarkbmarun we have merged 17.5k commits to openstack since october of last year22:03
marunfungi: there is a real benefit here, whether you are personally affected or not22:03
marunclarkb: at what cost?22:04
*** boden has joined #openstack-infra22:04
clarkbmarun: I am not sure what you are using "cost" to measure here22:04
marunclarkb: you can pull numbers all you want, but merging code in openstack SUCKS22:04
clarkbmarun: but I thought it was that code cannot merge22:04
marunclarkb: developer productivity22:04
*** boden has quit IRC22:04
clarkbmarun: merging 17.5k commits in one year is not productive?22:04
fungimarun: i thought the current pain point was that the projects were broken and needed more people fixing those bugs in them? we'd effectively go to (rather a lot of) effort to make things easier for devs who aren't contributing to fixing that pain22:04
marunclarkb: being roadblocked repeatedly by long gate queues22:04
*** boden has joined #openstack-infra22:04
marunfungi: we can't fix bugs particularly fast if a) we have limited gate resources and b) those gate resources are not being applied as effectively as they could be22:05
marunfungi: this is part of the problem, too22:05
marunit's multifaceted22:05
*** boden has quit IRC22:05
*** boden has joined #openstack-infra22:05
fungimarun: yes, there are at least 10x as many facets to the problem as you're seeing from your perspective too, and we are already up to our eyeballs trying to solve them22:05
*** salv-orlando has joined #openstack-infra22:06
*** boden has quit IRC22:06
openstackgerritClark Boylan proposed a change to openstack-infra/project-config: Port build-image.sh from old config repo  https://review.openstack.org/12425422:06
*** bhuvan has joined #openstack-infra22:06
clarkbanteaya: ^ is the fix that 124254 needed. you should be good to remove those files from the other repo22:06
fungiso forgive me if i don't see avoiding running unit tests on non-code changes as a particularly high priority feature22:06
*** boden has joined #openstack-infra22:06
marunfungi: So you think it's ok to ignore my complaints because *you* don't think they are important.22:07
marungreat22:07
fungii don't disagree that there could be some gain22:07
stevemarcan I get one more review on https://review.openstack.org/#/c/125202/ i'm eager to get some test code in22:07
* marun gives up22:07
*** boden has quit IRC22:07
*** boden has joined #openstack-infra22:07
fungimarun: i will personally attempt to prioritize it appropriately along with everything else which i also know could possibly help22:07
marunfungi: I'm not asking for you to do work22:08
*** boden has quit IRC22:08
fungiand alongside the higher priority things which i know are currently broken22:08
*** boden has joined #openstack-infra22:08
marunfungi: I'm asking for some appreciation that developer productivity matters and the impact of the current gate structure may not be the best tradeoff for openstack.22:08
*** bknudson has quit IRC22:08
marunfungi: where we go from there is entirely a separate issue22:08
*** boden has quit IRC22:09
*** krtaylor has joined #openstack-infra22:09
*** boden has joined #openstack-infra22:09
fungimarun: if you think for even a moment that we, as a group, are not taking the efficiency of our community development as the absolute top priority, we may as well all give up and leave22:09
fungibecause honestly, we're all failing if that's the case22:09
*** mbacchi has quit IRC22:09
*** boden has quit IRC22:09
marunfungi: I think it's on both sides22:10
*** boden has joined #openstack-infra22:10
marunfungi: I recognize that infra is working impossibly hard to do an impossible job22:10
fungii can't actually remember one thing i've worked on which isn't specifically focused on smoothing and improving throughput for the development of openstack22:10
marunfungi: I do think, though, that there is an insufficient appreciation for developer concerns.22:10
clarkbfungi: re https://review.openstack.org/#/c/125186/ any opposition to self approving? was hoping mordred would weigh in but its a simple change22:10
*** boden has quit IRC22:10
*** mattoliverau has joined #openstack-infra22:10
fungimarun: i too am a developer22:11
marunfungi: And talking about this stuff here is an attempt to chart a different course in the future.22:11
*** boden has joined #openstack-infra22:11
marunfungi: let's be clear22:11
marunfungi: do you work on critical-path features on a core openstack project?22:11
*** boden has quit IRC22:11
*** david-lyle_ has joined #openstack-infra22:11
fungimarun: aside from fixing bugs in them sometimes, no22:11
*** david-lyle has quit IRC22:11
*** boden has joined #openstack-infra22:12
dstufftfungi: wrt to your post about getting rid of external things in openstack - https://www.reddit.com/r/Python/comments/2hvi5m/looking_for_the_cdecimal_module_and_bytereeforg/ (cdecimal author is one of the main people against mandating things uploaded to PyPI)22:12
dstufft:||22:12
marunfungi: I'm not questioning your technical capabilities.  I'm saying the pain I'm seeing people facing in projects like neutron is crazy.22:12
fungiand assuming you don't consider security support critical-path22:12
*** boden has quit IRC22:12
*** david-lyle has joined #openstack-infra22:12
*** mattoliverau has quit IRC22:12
*** boden has joined #openstack-infra22:12
marunfungi: that is absolutely critical-path22:12
*** thedodd has quit IRC22:13
fungiand trust me, when we've got embargoes security fixes with advisory publication deadlines and can't get them merged because of bugs in openstack and poorly written tests, i'm very much aware of that pain22:13
marunfungi: but presumably you can fast-track stuff that is so important22:13
marunfungi: but for those whose constant pain is trying to get things through the gate?22:13
*** boden has quit IRC22:13
marunfungi: something has to give22:13
fungionly to a point. we still won't merge security fixes if we can't get them to pass tests22:13
marunfungi: I want things to be better.  I want to help.22:13
*** boden has joined #openstack-infra22:13
*** mattoliverau has joined #openstack-infra22:13
*** boden has quit IRC22:14
fungime too22:14
*** david_lyle__ has joined #openstack-infra22:14
openstackgerritA change was merged to openstack-infra/nodepool: Properly reference nodepool config in image update  https://review.openstack.org/12518622:14
mordredme too - even when I get unreasonably ragey22:14
*** boden has joined #openstack-infra22:14
clarkbfungi: thank you for approving22:14
fungidstufft: too fun22:15
*** boden has quit IRC22:15
fungidstufft: we probably shouldn't be using cdecimal22:15
*** boden has joined #openstack-infra22:15
*** boden has quit IRC22:15
*** david-lyle_ has quit IRC22:16
fungidstufft: and looks like we're not ;)22:16
mordreddstufft: I completely do not grok the people who are oppoised to pypi upload22:16
*** boden has joined #openstack-infra22:16
dstufftmordred: especially when the thing they don't want to upload is BSD licensed? ;)22:16
*** boden has quit IRC22:16
*** yamahata has quit IRC22:17
dstufftNormally one of the excuses is that PyPI's ToS are too onerous22:17
*** david-lyle has quit IRC22:17
fungidstufft: i mainly wanted to point out to the detractors who claim the developer community will abandon pypi if it dictates sane release standards that there are also parts of the python community who could be driven away if it doesn't22:17
*** boden has joined #openstack-infra22:17
*** yamahata has joined #openstack-infra22:17
*** rmcall has joined #openstack-infra22:17
*** boden has quit IRC22:17
fungidstufft: and that perhaps the loss of one of those two groups would be less detrimental than the loss of the other22:17
dstufftbecause they require you grant us an unlimited royality free <insert legal crap> license to redistribute forever and for anyone who uses PyPI to redistribute forrever22:17
*** dmsimard is now known as dmsimard_away22:18
*** boden has joined #openstack-infra22:18
mordredegads. no. anything but that22:18
*** boden has quit IRC22:18
dstufft(notably there is no legal requirement that anyone be allowed to _use_ what you upload to PyPI, or modify it, only distribute)22:18
dstufftthe legal clause is entirely "make sure you can't sue the PSF and you can't sue mirror operators"22:19
fungiso they'd like to somehow be able to retroactively revoke the abiliaty for anyone to redistribute things uploaded to pypi? that's a little pathological22:19
dstufftfungi: funny thing, PyPI is growing in use22:19
dstufftfungi: yes22:19
dstufftthey want to do that22:19
*** boden has joined #openstack-infra22:20
* fungi tries to figure out how that would even work22:20
*** david_lyle__ has quit IRC22:20
dstufftplus one of the people against it is in germany and he releases crypto stuff22:20
clarkbfungi: thoughts on restarting nodepool shortly?22:20
dstufftand he thinks the export laws in the US still matter22:20
fungiclarkb: we can restart it now22:20
*** boden has quit IRC22:20
clarkbfungi: I can do it /me double checks new code is installed22:20
*** dims_ has joined #openstack-infra22:20
clarkbit appears to be installed22:20
*** boden has joined #openstack-infra22:20
clarkbfungi: no hope in a gracefull restart ya?22:20
fungistop it and then start it22:21
clarkbkk /me does this22:21
*** boden has quit IRC22:21
fungiit will stop cleanly and quickly22:21
fungiin my experience22:21
dstufftfungi: this is what the total downloads per week looks like on PyPI since Jan 2014 http://d.stufft.io/image/1f0j0k2u1r0Q22:21
fungirestart seems to hang for reasons i've not had time to debug22:21
dstufftfungi: I think that we're doing *something* right22:21
clarkbdone22:21
jheskethMorning22:21
*** boden has joined #openstack-infra22:21
*** r1chardj0n3s_afk is now known as r1chardj0n3s22:21
clarkbI think I will wait for tomorrow and the subsequent image builds before changing hpcloud* over to the dib devstack trusty image22:22
zarojhesketh: morning22:22
*** boden has quit IRC22:22
fungidstufft: yep, i just hate seeing people try to stand in the way of progress because they're afraid of change22:22
*** boden has joined #openstack-infra22:22
*** dims_ has quit IRC22:22
*** boden has quit IRC22:23
clarkbmarun: so going back to productivity. The measure you would like to see used is number of rechecks or time to merge post approval?22:23
zarojhesketh: are you still working on this one? https://review.openstack.org/#/c/107724/22:23
*** dkehnm has quit IRC22:23
*** gondoi is now known as zz_gondoi22:23
*** dims_ has joined #openstack-infra22:23
clarkbmarun: trying to understand what in particular you are finding problematic22:23
marunclarkb: amount of time to merge22:23
*** boden has joined #openstack-infra22:23
fungibecause some devs might stop using pypi if you tell them they can't reuse version numbers for new package uploads22:23
*** hdd has quit IRC22:23
clarkbmarun: in total since proposed?22:23
*** jd__ has quit IRC22:23
*** dmsimard_away is now known as dmsimard22:23
marunclarkb: no, since approved22:23
clarkbkk22:23
*** dims has quit IRC22:23
clarkbjogo: did you run numbers like that for nova at some point? was it compiled anywhere?22:23
jogomarun: that is measurable22:23
clarkbjogo: curious if you have ways to collect that already22:24
*** boden has quit IRC22:24
jogomarun: I didn't do that part of the tseting22:24
marunclarkb: assuming we can approve before the check has come back and that the gate won't start until it comes back22:24
*** boden has joined #openstack-infra22:24
clarkbmarun: you can22:24
jogobut I can share my scripts in github so you can add time to merge since first approval22:24
marunclarkb: isn't that disabled during peak times?22:24
*** jd__ has joined #openstack-infra22:24
clarkbmarun: no22:24
jheskethzaro: yep, still keen for it to merge. I'll take a look at it today22:24
*** boden has quit IRC22:24
clarkbmarun: you always need a +122:24
clarkbmarun: but you can approve before you have that +1 and it will go into the gate after the +1 from jenkins arrives22:25
marunclarkb: I seem to remember being warned not to approve until a check passed around FF22:25
*** boden has joined #openstack-infra22:25
jhesketh(I didn't realise there were comments)22:25
marunclarkb: am I misremembering?22:25
clarkbmarun: I think that was last cycle before we implemented the clean check requirements22:25
*** boden has quit IRC22:25
jogoclarkb: so this is one big area of miscommunication: what the current gate behaviors are22:25
marunclarkb: I swear I recall being told that had been turned off during FF for some reason22:26
*** boden has joined #openstack-infra22:26
clarkbmarun: we can check git logs but pretty sure it wasn't22:26
marunclarkb: arg.  the whole neutron team was behaving like it was22:26
marun:/22:26
marunwe were holding off on approval until the check approval came in22:26
*** boden has quit IRC22:26
clarkbjogo: I think part of the problem there is that the number of devs that understand gating is small22:26
*** boden has joined #openstack-infra22:26
clarkbjogo: so when we try to communicate behaviors it creates more confusion (granted tis a thing that should be solved)22:27
marunsdague: Do you remember anything about the clean check requirements being relaxed around FF for some reason22:27
marun?22:27
clarkbmarun: they were relaxed but not removed22:27
*** boden has quit IRC22:27
marunclarkb: what does 'relaxed' imply?22:27
sdaguemarun: freshness checks were removed22:27
*** boden has joined #openstack-infra22:27
clarkbmarun: we stopped running the <24 hour old +1 requirement22:27
clarkbmarun: whcih is different than requiring clean check22:28
jogoclarkb: agreed, a doc that we can point folks to would be useful22:28
marunsdague: ah, ok.  and I guess we were being careful around that22:28
sdaguewhich I did explain in an email, and no one understood what it meant22:28
*** boden has quit IRC22:28
marunsdague: I'd just forgotten22:28
sdaguejogo: seriously, a doc will not help22:28
clarkbjogo: ^ that is what I am talking about22:28
clarkbgating is complex22:28
clarkband most devs don't understand the simple case22:28
clarkbI have even had people ask why you would bother testing code before it merges...22:28
*** boden has joined #openstack-infra22:28
marunclarkb: Well, it works for google...22:28
fungiclarkb: it's a reasonable question. many (most?) projects don't22:29
* marun stops trolling22:29
*** boden has quit IRC22:29
clarkbmarun: iirc google does gating on some projects22:29
jogosdague: making the current set of gate rules etc hard to discover isn't helping22:29
marunclarkb: they always run lots of tests22:29
*** flaviof is now known as flaviof_zzz22:29
clarkbdidn't google go to jeblair at linux con and say hey we do that22:29
*** boden has joined #openstack-infra22:29
marunclarkb: but they have more than one path to stability22:29
*** boden has quit IRC22:30
clarkbfungi: I think the fact that many projects don't is independent of "testing your code before merging it is a good thing"22:30
marunclarkb: pre- and post-merge have their place22:30
clarkbmarun: agreed22:30
*** boden has joined #openstack-infra22:30
marunclarkb: it's questionable that our attempts to catch everything pre-merge are worth the cost22:30
marunclarkb: a 2-pronged approach might also be viable22:30
*** boden has quit IRC22:30
clarkbmarun: we have argued for 2 pronged...22:31
marunclarkb: but we need better per-project testing of course :/22:31
clarkbmarun: one of the things we want to happen is to stop testing postgres and cells and all these funky combos on every change22:31
*** boden has joined #openstack-infra22:31
marunclarkb++22:31
*** praneshp has quit IRC22:31
clarkband shift that to more of a post workflow22:31
*** ZZelle_ has quit IRC22:31
clarkbso that interested parties have the data they need to keep those features working without impacting the stability of common setups22:31
*** boden has quit IRC22:31
clarkbbut that hasn't seemed to have moved very far since the idea happened22:32
*** boden has joined #openstack-infra22:32
*** boden has quit IRC22:32
*** boden has joined #openstack-infra22:33
marunclarkb: how about those jobs are moved out of the integrated gate?22:33
clarkbmarun: thats basically what would happen22:33
jogoclarkb: speaking of post merge tests. what is the progress on that?22:33
*** boden has quit IRC22:33
clarkbjogo: I dno't think anyone has started working on it. we probably need bigger buy in22:33
marunclarkb: and then the interested parties can devote resources to making post-merge happen22:33
clarkbbefore we tell postgres and cells we are changing their workflow :)22:33
*** boden has joined #openstack-infra22:34
marunclarkb: right now they have no incentive - they already have what they want22:34
clarkbmarun: right22:34
*** boden has quit IRC22:34
clarkbsdague: you started that proposal, did it die early?22:34
clarkbsdague: maybe we should pick that thread up again and say look we are doing this early in kilo22:34
sdagueI started it22:34
jogowhat kind of buy in is needed?22:35
*** boden has joined #openstack-infra22:35
clarkbjogo: we should probably make sure that ptls are on board22:35
sdaguejogo: no buy in needed, just workers22:35
jogoI think I can get nova-core to agree we want that22:35
jogosdague: ahh thats what I thought22:35
clarkbsdague: that works for me too :)22:35
*** boden has quit IRC22:35
sdaguehonestly, there is a ton of work that needs to happen for it22:35
clarkbsdague: ya its not a simple change22:35
sdagueand, I'll be honest, I got burnt out on it22:35
*** boden has joined #openstack-infra22:35
jogosdague: yeah in germany the list of steps was pretty big22:35
*** jgrimm is now known as zz_jgrimm22:36
*** boden has quit IRC22:36
*** homeless has quit IRC22:36
*** praneshp has joined #openstack-infra22:36
sdaguebecause the other thing is, when you spend time looking at gate issues a lot, what you actually want to do is remove +2 authority from all the core teams22:36
*** boden has joined #openstack-infra22:36
marunsdague: I can imagine that sentiment22:36
fungithe rate at which we merge new bugs seems to be at least proportional to the rate at which we merge anything22:37
*** boden has quit IRC22:37
marunsdague: I see stuff merge in Neutron that shouldn't, all the time.  :/22:37
sdagueespecially when you have critical bugs that no one looks at22:37
sdaguethat are ruining the gate for everyone22:37
*** homeless has joined #openstack-infra22:37
sdagueand no one cares22:37
*** boden has joined #openstack-infra22:37
sdaguesee... -meeting from an hour ago22:37
clarkbmarun: so the 2 pronged approach may be a good first step on reducing that friction. Then as the functional test suites become more robust (and actually exist) we can revisit the stop running tempest a lot question22:37
*** boden has quit IRC22:38
fungiand it's amusing/sad to look at timelines of varying levels of our breakage, to see that as soon as we fix enough bugs to make it faster for project to land new changes, they merge in new bugs more quickly to get back to the same pain threshold22:38
*** boden has joined #openstack-infra22:38
marunfungi, sdague: there is little incentive for people to write better quality features when they aren't directly responsible for fixing bugs. :/22:38
*** boden has quit IRC22:39
fungimarun: that was the philosophy behind pre-merge testing, but it doesn't account for nondeterministic bugs22:39
*** boden has joined #openstack-infra22:39
marunThe only way around this from my perspective is providing sufficient test infrastructure in-project and then mandating that all project-specific testing be present pre-merge22:39
*** david-lyle has joined #openstack-infra22:39
*** boden has quit IRC22:39
*** _buzz has quit IRC22:39
sdagueright, and it definitely didn't account for developer behavior of not giving a crap about the project they work on and running 'recheck' 10 times to land their feature because their project manager wanted it in this release22:40
marunWe're making progress, slowly, on this in neutron.  At least functional tests are possible.22:40
*** boden has joined #openstack-infra22:40
*** boden has quit IRC22:40
*** boden has joined #openstack-infra22:41
*** ryanpetrello has quit IRC22:41
*** boden has quit IRC22:41
*** tsg has quit IRC22:41
*** david-lyle_ has joined #openstack-infra22:41
*** boden has joined #openstack-infra22:42
marunsdague, fungi, clarkb: Are most gate-blocking bugs race conditions?22:42
clarkbmarun: not always, but often yes22:42
*** boden has quit IRC22:42
*** datsun180b has quit IRC22:42
*** boden has joined #openstack-infra22:43
clarkba non race condition that was making the gate run really slowly had to do with keystone performance under a single eventlet working process22:43
clarkbit wasn't quite blocking but added 30 minutes to test run time22:43
clarkband no one noticed until I said hey guys whats going on here...22:43
*** boden has quit IRC22:43
marunclarkb: So, bugs that slipped through despite all the integrated jobs...22:43
fungithe longer-lived ones are race conditions and/or other similar nondeterministic bugs in openstack or in the way tests are written22:43
marunclarkb: yikes22:43
*** boden has joined #openstack-infra22:43
*** armax has quit IRC22:44
clarkbmarun: sort of. in many cases the issues were caught at least one in check or gate22:44
clarkbmarun: then they get rechecked until they merge22:44
*** boden has quit IRC22:44
clarkbmarun: we have seen that happen on a few of these bugs22:44
*** david-lyle has quit IRC22:44
fungithere are also plenty of "gate-breaking" bugs which are due to external causes (dependency changes, incidents at service providers impacting our ci systems, et cetera) but those tend to get identified and addressed fare more quickly22:44
*** boden has joined #openstack-infra22:44
sdaguemarun: ... if only I'd given a talk on this subject ... ;)22:44
*** boden has quit IRC22:45
marunsdague: I'm wondering - if our biggest problem is race conditions that slip through despite our best efforts.22:45
sdaguemarun: yes, because statistics are against you22:45
*** boden has joined #openstack-infra22:45
*** hdd has joined #openstack-infra22:45
fungiand, irony of ironies, many of the longer-lived sources of failures in providers is *actually* nondeterministic bugs in openstack22:45
clarkbya we have seen manifestations of git bugs in our providers more than once22:45
sdaguegiven a complex system, if a failure were to happen 10% of the time22:46
clarkbs/git/gate/22:46
*** boden has quit IRC22:46
fungithings like loss of connectivity from workers to git mirrors, pypi mirrors, et cetera... actually openstack bugs impacting our providers22:46
*** yamahata has quit IRC22:46
*** boden has joined #openstack-infra22:46
marunfungi: yikes22:46
*** dane_leblanc has quit IRC22:46
*** beecee has joined #openstack-infra22:46
*** boden has quit IRC22:46
fungibugs which have slipped through testing, and yes, sometimes bugs where we already have failure log signatures and nobody's had time to run down the underlying causes22:47
marunthat implies, though, that the integrated gate serves more to slow down merging of new patches (new bugs) more than anything22:47
*** boden has joined #openstack-infra22:47
marunSince a good proportion of the problems have nothing to do with the patches at all22:47
*** boden has quit IRC22:47
*** ryanpetrello has joined #openstack-infra22:47
fungiwell, i'd wager that if we weren't doing the current base level of testing, providers wouldn't be using this at all22:47
marunAnd race conditions are as likely to slip through as anything22:47
marunfungi: I'm not suggesting that testing isn't required22:48
*** boden has joined #openstack-infra22:48
fungirace conditions and similar nondeterministic bugs are what slip through, yes, as well as bugs in code which isn't covered by tests of course22:48
marunfungi: sure.22:48
*** boden has quit IRC22:48
clarkbmarun: except that in many cases the patches that introduce the race actually failed tests at least once22:48
*** boden has joined #openstack-infra22:49
clarkbbecause the net is wide22:49
marunfungi: it would seem, though, that we could accomplish a similar result by artificially limiting the merge rate and running a crapload of integrated job post-merge22:49
marunI'm not saying that's a good idea22:49
marunonly that the result would effectively be the same22:49
*** boden has quit IRC22:49
*** boden has joined #openstack-infra22:50
sdaguemarun: there are definitely lots of incremental ways to make this better22:50
sdaguebut they mostly need people to work on them22:50
marunclarkb: it can be hard for the average developer to discern a difference between a transient, infra bug, an unrelated race condition, or a failure they themselves are causing22:50
*** boden has quit IRC22:50
marun:/22:50
clarkbmarun: I would say that is true for a chunk of them, but most peopel seem to completely ignore test results22:50
*** boden has joined #openstack-infra22:50
*** JayJ has quit IRC22:51
sdaguemarun: I agree that debugging openstack is hard22:51
sdaguewe should fix that22:51
*** boden has quit IRC22:51
* marun waves his magic wand22:51
clarkbI know I am guilty of only looking at the SUCCESS v FAILURE reported back to gerrit22:51
maruntada!22:51
*** boden has joined #openstack-infra22:51
sdaguehonestly, I think there is value in your idea about limitted testing support in zuul, honestly, I'd like it for docs/ trees (that's where I originally brought up the idea)22:52
*** boden has quit IRC22:52
sdaguebut ... someone needs to hack on it22:52
*** boden has joined #openstack-infra22:52
sdagueso if you think it's a top fix to dev productivity, raise your hand and dive in, and I'm sure people will point you in the right direction22:53
*** boden has quit IRC22:53
openstackgerritClark Boylan proposed a change to openstack-infra/config: Add devstack-precise-dib image to nodepool  https://review.openstack.org/12522022:53
*** baoli has joined #openstack-infra22:53
*** boden has joined #openstack-infra22:53
marunsdague: will do22:53
marunsdague: I22:53
fungii would argue (and have) that troubleshooting integration test failures is basically the same as if you were a one-person operations team at a service provider troubleshooting your cloud falling over22:53
fungithe range of possible ways it can go wrong is basically the same22:53
*** boden has quit IRC22:53
marunfungi: Can I assume that this is a bad thing?22:54
marunfungi: or is it necessary complexity?22:54
sdaguewell, we don't make it easy for people22:54
*** boden has joined #openstack-infra22:54
sdagueseriously, staring at neutron logs for the last 2 hrs here22:54
sdaguewe do not make it easy for people22:54
marunsdague: arguably our lack of project-focused tests is at fault here22:54
fungii think it's necessary complexity, but i also think we as openstack are in a better position to make those bugs more shallow than $random_operator22:54
marunsdague: most issues should not be debugged in an integrated env22:54
*** boden has quit IRC22:54
marunor rather, should be able to be debugged in isolation22:55
*** boden has joined #openstack-infra22:55
sdaguemarun: so I only partially agree22:55
*** boden has quit IRC22:55
marunsdague: me too :)22:55
sdagueif things go wrong, the system should be giving you enough info to figure it out22:56
*** boden has joined #openstack-infra22:56
sdagueespecially when we are running at DEBUG22:56
marunsdague: I kind of agree22:56
fungisome issues yes. some issues only surface under integration because they're bugs in how the projects interact with one another22:56
marunsdague: but working at too high a level by choice seems unwise22:56
*** boden has quit IRC22:56
marunsdague: if we're developers, we can choose the level we want to work at22:56
sdaguesure22:56
fungidefining and testing those interfaces would make a huge difference there, but we're a very long way from that being feasible from what i've seen22:56
marunsdague: operators, though, we have to cover that demographic too.  no question22:57
sdaguebut we also need to make a system that operators have any hope of debugging22:57
*** boden has joined #openstack-infra22:57
sdaguethe gate is basically "lets play operator"22:57
*** koolhead17 has quit IRC22:57
*** boden has quit IRC22:57
*** andreaf has quit IRC22:57
marunsdague: that being the case, I don't think that every single patch should be triggering the 'operator' switch22:57
*** boden has joined #openstack-infra22:57
*** stevemar has quit IRC22:57
*** andreaf has joined #openstack-infra22:58
marunsdague: there is pain in the 'converge towards stability post-merge' approach22:58
*** jergerber has quit IRC22:58
marunsdague: no question22:58
*** boden has quit IRC22:58
marunsdague: but gating everyone on the weakest member is as bad an idea in software as it is in chains22:58
*** boden has joined #openstack-infra22:58
*** boden has quit IRC22:59
marunreliable distributed software attempts to limit failure domains22:59
*** koolhead17 has joined #openstack-infra22:59
*** koolhead17 has quit IRC22:59
sdaguewell, under that assumption, we'd just delete neutron as a project22:59
marunour gate process conflates them22:59
*** boden has joined #openstack-infra22:59
marunsdague: I wouldn't argue :)22:59
openstackgerritMichael Krotscheck proposed a change to openstack-infra/storyboard-webclient: URL Util now maps full install path.  https://review.openstack.org/12522222:59
*** boden has quit IRC23:00
*** boden has joined #openstack-infra23:00
marunsdague, clarkb, fungi, mordred: I have to run.  Thank you for the constructive conversation, as always.23:00
*** AndroUser2 has joined #openstack-infra23:00
openstackgerritMichael Krotscheck proposed a change to openstack-infra/storyboard-webclient: Subscription  https://review.openstack.org/10400423:00
*** boden has quit IRC23:00
marunand for bearing with me23:01
openstackgerritClark Boylan proposed a change to openstack-infra/config: Switch hpcloud devstack-trusty to dib image  https://review.openstack.org/12522323:01
openstackgerritClark Boylan proposed a change to openstack-infra/config: Put devstack-trusty-dib image in rax  https://review.openstack.org/12522423:01
clarkbfungi: ^ I don't think I am ready to merge those yet. I want to see image updates working again tomorrow first23:01
clarkbfungi: but that gievs you an idea of what I am thinking for the continung dibification of stuff23:01
*** boden has joined #openstack-infra23:01
*** boden has quit IRC23:01
*** boden has joined #openstack-infra23:02
*** david-lyle_ has quit IRC23:02
*** boden has quit IRC23:02
*** boden has joined #openstack-infra23:03
*** boden has quit IRC23:03
*** esker has quit IRC23:03
*** boden has joined #openstack-infra23:04
*** koolhead17 has joined #openstack-infra23:04
*** weshay has joined #openstack-infra23:04
*** dkranz has quit IRC23:04
*** koolhead17 has quit IRC23:04
*** boden has quit IRC23:04
openstackgerritK Jonathan Harker proposed a change to openstack-infra/nodepool: Reset the client object before creating the image  https://review.openstack.org/7494323:04
*** boden has joined #openstack-infra23:04
*** boden has quit IRC23:05
*** marun has quit IRC23:05
clarkbsdague: fungi: so I wonder if debugging the gate being so much like what happens in ops trenches is partially to blame for devs not debugging as much as we like?23:05
*** boden has joined #openstack-infra23:05
clarkbI think its the proper way to do things so that we feel the pain of operators23:05
clarkbbut it could explain part of the "hardness" of it23:06
clarkbops is different23:06
*** boden has quit IRC23:06
*** boden has joined #openstack-infra23:06
*** boden has quit IRC23:07
clarkbdebugging large systems without running an attached debugger is very much an ops thing23:07
krotscheckCan I get a sanity confirmation of something? Go to this review, click on ‘gate-storyboard-webclient-js-draft’, and try to log in?23:07
krotscheckhttps://review.openstack.org/#/c/125222/23:07
*** boden has joined #openstack-infra23:07
clarkbdevs want to have stuff in as narrow a focus as possible with debugger attached23:07
clarkbkrotscheck: so I need to start logged out?23:08
*** boden has quit IRC23:08
clarkboh wait23:08
clarkbyou want me to login to the thing linked gotcha23:08
*** boden has joined #openstack-infra23:08
krotscheckclarkb: RIght23:08
krotscheckclarkb: Looks like it still can’t post because of a CORS header issue, but I can fix that...23:08
clarkbit let me in23:09
jogoclarkb: I do think that if all OpenStack devs had to deploy OpenStack from scratch (re: no devstack packages and such are OK) and debug an install, things would be better23:09
*** boden has quit IRC23:09
krotscheckclarkb: AWESOME23:09
*** boden has joined #openstack-infra23:09
*** boden has quit IRC23:09
clarkbjogo: well I think it may just be that as individuals withd ifferent skill sets we thinkdifferentl23:09
clarkbjogo: I don't tend to have problems jumping in and doing hands off debugging based on logs and such ebcause I do lots of ops23:10
clarkbjogo: its the way we do things23:10
*** boden has joined #openstack-infra23:10
clarkbbut if your background is always being attached to a debugger and needing to debug based on live events doing it the other way can be hard23:10
*** boden has quit IRC23:10
*** baoli has quit IRC23:10
mtreinishjogo: I've done it, it wasn't too bad for me, only took me about a week...23:10
clarkbmtreinish: jogo: I did it in an afternoon with diablo...23:11
mordredI add print('dammit') in a lot23:11
*** boden has joined #openstack-infra23:11
mordredit's not super scientific23:11
*** sarob is now known as sarob_away23:11
jogomtreinish: how recent was that?23:11
clarkbannegentle_'s install guide was great :)23:11
jogomtreinish: that it took a week23:11
mtreinishjogo: the week before I left IBM23:11
*** boden has quit IRC23:11
*** weshay is now known as weshay_afk23:11
jogomtreinish: wow,  so fairly recently23:11
*** MaxV has quit IRC23:12
clarkband having functional tests where you can attach directly to those live events will help significantly23:12
*** boden has joined #openstack-infra23:12
openstackgerritMichael Krotscheck proposed a change to openstack-infra/storyboard: Added Content-Type header to CORS  https://review.openstack.org/12522723:12
clarkbor at least I think that is what my theory is becoming23:12
*** atiwari has joined #openstack-infra23:12
*** boden has quit IRC23:12
mtreinishclarkb: well I did it for a rack and a half23:12
clarkbbecause that is an env that devs are comfortable in. But we shouldn't forget indirect debugging because operators everywhere deal with our tools that way23:12
*** sarob_away is now known as sarob23:12
clarkbmtreinish: ah, I only did a one node install23:12
*** boden has joined #openstack-infra23:12
mtreinishjogo: I didn't say I did a good job though :) a bunch of things were horribly misconfigured23:13
mtreinishbut it worked...23:13
jogomtreinish: haha23:13
*** achanda has joined #openstack-infra23:13
*** boden has quit IRC23:13
jogomtreinish: if it took the PTL of QA a week to just get it working ....23:13
*** dims_ has quit IRC23:13
jogothat is not good23:13
*** boden has joined #openstack-infra23:13
mtreinishjogo: oh and flatdhcp n-net I had enough sense to avoid trying neutron23:14
jogomtreinish: haha wise23:14
clarkbjogo: it isn't but honestly it takes that long to install a vmware cluster of a rack and a half. stuff is complex23:14
*** boden has quit IRC23:14
*** AndroUser2 has quit IRC23:14
*** boden has joined #openstack-infra23:14
jogoclarkb: hmm I didn't realize vmware takes that long as well23:14
*** IAX has quit IRC23:15
mordredclarkb: I wonder how long a rack of infra would take23:15
*** boden has quit IRC23:15
clarkbjogo: people like to say it doesn't take that long, but when I did it a few years back it took ages23:15
clarkbjogo: networking is a big hang up there too23:15
*** armax has joined #openstack-infra23:15
*** boden has joined #openstack-infra23:15
jogoclarkb: but I do think the act of having more developers feel the operator/usability pain is a good thing23:15
clarkbmordred: lifeless did it in what a week?23:15
jogoone way or onother23:15
*** stevemar has joined #openstack-infra23:16
*** boden has quit IRC23:16
mtreinishclarkb: yeah networking was my issue too. Luckily that was a replacement setup so I had learned all the tough lessons already23:16
*** boden has joined #openstack-infra23:16
mtreinishit took me a month to work through some of that the first time (with only 4 nodes)23:16
*** boden has quit IRC23:16
*** baoli has joined #openstack-infra23:16
*** loquacities has joined #openstack-infra23:17
*** boden has joined #openstack-infra23:17
*** boden has quit IRC23:17
clarkbtl;dr networking is hard23:17
clarkbwhich is one of the reasons I like jeblair's philosophy of computers should just talk to each other :)23:18
*** boden has joined #openstack-infra23:18
clarkbit reduces a lot of complexity when you say that proxies and intermediate firewalls should diaf23:18
*** boden has quit IRC23:18
*** boden has joined #openstack-infra23:19
*** boden has quit IRC23:19
*** boden has joined #openstack-infra23:20
*** armax has quit IRC23:20
*** boden has quit IRC23:20
*** armax has joined #openstack-infra23:20
*** boden has joined #openstack-infra23:20
*** dkranz has joined #openstack-infra23:20
*** boden has quit IRC23:21
fungiclarkb: jogo: as a point of reference, vce was (presumably still is) a large vmware integrator. they used an orchestration tool from emc, the name of which escapes me, to install vmware starting from bare metal, more or less turn-key... deploying several racks took on the order of a couple days of the orchestration software chugging nonstop23:21
*** boden has joined #openstack-infra23:21
*** boden has quit IRC23:22
*** boden has joined #openstack-infra23:22
*** sarob is now known as sarob_away23:22
clarkbfungi: we did dense blade chassis at a slower rate mostly manually23:22
*** bswartz has joined #openstack-infra23:22
clarkbbecause automation didn't exist for various things in the stack23:23
*** MarkAtwood has joined #openstack-infra23:23
*** boden has quit IRC23:23
*** boden has joined #openstack-infra23:23
jheskethclarkb: if you have time, I need some help debugging zuul swift stuff23:23
fungiright. point being, fully automated, days to install a few racks. human intervention would obviously have multiplied that several times over23:23
clarkbjhesketh: sure23:23
clarkbjhesketh: I am mostly done with dib for the day23:23
*** boden has quit IRC23:23
jheskethclarkb: awesome, thanks :-)23:24
jheskethclarkb: basically Jenkins isn't getting the correct swift values23:24
jheskethie it hasn't got any env vars23:24
*** boden has joined #openstack-infra23:24
fungii am mostly done for the day, with the usual feeling of having accomplished none of the things i set out to get done. there's always tomorrow!23:24
jheskethclarkb: are you able to see how swift is configured in zuul.conf?23:24
clarkbjhesketh: is this happening with every job?23:24
jheskethclarkb: as far as I can tell, yes23:24
clarkbjhesketh: I can check zuul.conf now23:24
*** boden has quit IRC23:24
*** boden has joined #openstack-infra23:25
*** boden has quit IRC23:25
*** _nadya_ has joined #openstack-infra23:25
*** boden has joined #openstack-infra23:26
*** boden has quit IRC23:26
*** boden has joined #openstack-infra23:27
lifelessclarkb: did what in a week?23:27
*** sarob_away is now known as sarob23:27
clarkblifeless: deployed a set of openstack infra in not infra23:27
*** boden has quit IRC23:27
clarkbfor that talk you ran through the things23:27
clarkbjhesketh: I don't see anyting amiss there23:27
*** boden has joined #openstack-infra23:27
lifelessclarkb: are you referring to tripleo-ci ? or the testing-cabal deploy of zuul* ?23:27
clarkblifeless: testing cabal23:28
*** boden has quit IRC23:28
openstackgerritK Jonathan Harker proposed a change to openstack-infra/nodepool: Reset the client object before creating the image  https://review.openstack.org/7494323:28
lifelessa week of evenings got me most of the way yeah23:28
lifelessclarkb: what is a 'rack of infra' though23:28
*** boden has joined #openstack-infra23:28
lifelessclarkb: are you suggesting an infra testing infra deployment?23:28
clarkblifeless: more just a rack somewhere that runs infra services23:29
*** boden has quit IRC23:29
clarkbthat rack may be virtual I guess23:29
*** boden has joined #openstack-infra23:29
*** _nadya_ has quit IRC23:30
*** boden has quit IRC23:30
*** boden has joined #openstack-infra23:30
clarkbjhesketh: have an example jenkins job link?23:31
*** boden has quit IRC23:31
jheskethclarkb: https://jenkins01.openstack.org/job/gate-project-config-layout/14/injectedEnvVars/?23:31
*** boden has joined #openstack-infra23:31
jheskethclarkb: actually I think we aren't setting up swift correctly in the layout file... let me check23:31
clarkboh that could be23:31
*** boden has quit IRC23:31
*** boden has joined #openstack-infra23:32
jheskethclarkb: yeah, that appears to be it.. I'll shoot through a fix23:32
clarkbjhesketh: kk23:32
*** boden has quit IRC23:32
*** boden has joined #openstack-infra23:33
*** dmsimard is now known as dmsimard_away23:33
*** boden has quit IRC23:33
*** zz_jgrimm is now known as jgrimm23:33
*** boden has joined #openstack-infra23:34
*** camunoz has joined #openstack-infra23:34
*** boden has quit IRC23:34
*** otter768 has joined #openstack-infra23:34
*** boden has joined #openstack-infra23:35
openstackgerritJoshua Hesketh proposed a change to openstack-infra/project-config: Enable the swift logs env vars for all jobs  https://review.openstack.org/12523223:35
*** boden has quit IRC23:35
jheskethclarkb: ^23:35
*** stevemar has quit IRC23:35
anteayaclarkb: thanks23:36
*** mriedem has joined #openstack-infra23:36
*** boden has joined #openstack-infra23:36
clarkbjhesketh: I think you need to update your local version of layout.yaml then repropose? that diff is huge23:37
*** boden has quit IRC23:37
clarkbjhesketh: and undoes AJaeger's changes from ealier fi I am reading correctly23:37
*** dmellado has joined #openstack-infra23:37
*** boden has joined #openstack-infra23:37
jheskethhmm, probably a git fail sorry23:37
clarkbnp :)23:37
*** boden_ has joined #openstack-infra23:38
*** boden has quit IRC23:38
openstackgerritJoshua Hesketh proposed a change to openstack-infra/project-config: Enable the swift logs env vars for all jobs  https://review.openstack.org/12523223:38
jheskethclarkb: okay, try that23:38
*** boden_ has quit IRC23:39
*** boden has joined #openstack-infra23:39
*** boden_ has joined #openstack-infra23:40
*** boden has quit IRC23:40
clarkbjhesketh: muc better. is there an existing swift: section that can be removed?23:40
*** boden_ has quit IRC23:40
jheskethclarkb: oh yeah, on the experimental job... but I'll shoot through another patch to remove all of that job when we no longer want it23:41
*** boden has joined #openstack-infra23:41
clarkbjhesketh: sounds good23:41
*** gus has joined #openstack-infra23:41
*** boden has quit IRC23:41
*** boden has joined #openstack-infra23:42
sdagueclarkb: how do you feel about - https://review.openstack.org/#/c/124390/ ?23:42
*** boden has quit IRC23:42
sdaguejhesketh: also, I fixed the hideci issues I believe - https://review.openstack.org/#/c/124632/23:42
clarkbsdague: reading the commit message  Ithink that is a good idea23:42
sdaguethat would be nice to get out there23:42
*** MaxV has joined #openstack-infra23:42
clarkblet me review it properly23:42
*** boden has joined #openstack-infra23:42
lifelessclarkb: isn't that what you have today?23:43
jheskethsdague: cool, I was just reading through it :-)23:43
lifelessclarkb: is the idea to get away from cloud providers?23:43
*** boden has quit IRC23:43
clarkblifeless: no no, I think mordred was just wondering since we were talking about how long it takes to install openstack23:43
clarkbor isntall a vmware cluster23:43
*** boden has joined #openstack-infra23:43
lifelessclarkb: cause, you have an infra infra account on the tripleo clouds23:43
lifelessah23:43
lifelessso yeah, depends heavily on what trunk is like23:44
lifelessright now for instance we're two months into hp2 and its still not up23:44
*** boden has quit IRC23:44
*** boden has joined #openstack-infra23:44
*** Sincler has quit IRC23:44
*** boden has quit IRC23:45
*** boden has joined #openstack-infra23:45
*** boden_ has joined #openstack-infra23:46
*** boden has quit IRC23:46
*** andreaf has quit IRC23:46
*** JayJ has joined #openstack-infra23:46
*** boden_ has quit IRC23:46
*** andreaf has joined #openstack-infra23:47
*** JayJ has joined #openstack-infra23:47
*** boden has joined #openstack-infra23:47
*** MaxV has quit IRC23:47
*** boden has quit IRC23:47
*** boden has joined #openstack-infra23:48
*** boden has quit IRC23:48
*** boden has joined #openstack-infra23:49
*** ivar-laz_ has joined #openstack-infra23:49
*** boden has quit IRC23:49
*** boden has joined #openstack-infra23:50
*** ivar-laz_ has quit IRC23:50
*** boden has quit IRC23:50
jheskethsdague: reviewed.. I think there's a bug with the way you're loading the comments23:50
*** boden has joined #openstack-infra23:50
*** ivar-laz_ has joined #openstack-infra23:50
*** ivar-lazzaro has quit IRC23:51
sdaguemtreinish: https://review.openstack.org/#/q/status:open+project:openstack-dev/devstack+branch:stable/icehouse+topic:no_screen,n,z23:51
sdaguethose are the devstack patches I need23:51
*** boden has quit IRC23:51
*** ivar-laz_ has quit IRC23:51
*** boden has joined #openstack-infra23:51
openstackgerritA change was merged to openstack-infra/project-config: Enable the swift logs env vars for all jobs  https://review.openstack.org/12523223:51
fungijhesketh: if that ^ breaks the world, i won't be around to help debug ;)23:52
*** boden has quit IRC23:52
clarkbfungi: way to put us in the hot seat :P23:52
mtreinishsdague: ok, I'll take a look23:52
*** ivar-lazzaro has joined #openstack-infra23:52
*** boden has joined #openstack-infra23:52
fungiclarkb: you can always revert!23:52
clarkbindeed23:53
*** boden has quit IRC23:53
clarkbsdague: do you want to fix the whitespace thing?23:53
*** boden has joined #openstack-infra23:53
jheskethfungi: heh, I can never help debug when I merge stuff23:53
clarkbin that zuul change?23:53
jheskethmakes it scary23:53
clarkbalso not sure how comfortable I am merging zuul chagnes without jeblair around that are not bug fixes23:53
clarkbbut that change seems reasonable23:53
fungiit was a zuul configuration change23:54
*** boden has quit IRC23:54
fungiso seemed fair game23:54
*** MarkAtwood has quit IRC23:54
clarkbfungi: that one is fine23:54
*** boden has joined #openstack-infra23:54
clarkbI am talking about https://review.openstack.org/#/c/124390/423:54
fungioh, heh no23:54
*** boden has quit IRC23:54
*** boden has joined #openstack-infra23:55
fungithough i do need to re-review that one23:55
clarkbsdague: particularly since we seem to not be doing the thing that jeblair suggested?23:55
*** boden_ has joined #openstack-infra23:56
*** boden has quit IRC23:56
*** boden_ has quit IRC23:56
*** boden has joined #openstack-infra23:57
sdagueclarkb: so I thought we were doing it23:57
mtreinishsdague: do those test files get run anywhere?23:57
*** baoli has quit IRC23:57
clarkbsdague: it?23:57
*** boden has quit IRC23:57
sdaguemtreinish: honestly, I don't know23:57
*** chuckC_ has quit IRC23:57
*** boden has joined #openstack-infra23:57
*** ivar-laz_ has joined #openstack-infra23:58
*** boden has quit IRC23:58
clarkbsdague: "I like the cache approach used here, but I think we should consider the query-string and filtering options in Joshuas change. See more detailed comments there." is the jeblair comment i refer to23:58
*** boden has joined #openstack-infra23:58
clarkbor am I misreading what a query-string is in this case?23:58
* clarkb digs more23:58
sdagueclarkb: so I think that's patch #223:59
*** boden has quit IRC23:59
sdaguejhesketh: yep, you are correct, that's what I get for listening to people about optimizing :)23:59
clarkbsdague: the project filter is missing in yours23:59
*** boden has joined #openstack-infra23:59

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!