Monday, 2015-03-30

*** achanda has quit IRC00:00
*** tnovacik has quit IRC00:03
*** baoli has joined #openstack-infra00:04
*** baoli has quit IRC00:04
*** salv-orl_ has quit IRC00:06
*** armax has quit IRC00:07
*** masoz has joined #openstack-infra00:22
*** sdake has joined #openstack-infra00:23
*** sdake has quit IRC00:26
*** tiswanso has joined #openstack-infra00:26
*** annegentle has quit IRC00:26
*** sdake has joined #openstack-infra00:26
*** annegentle has joined #openstack-infra00:29
*** tiswanso has quit IRC00:30
openstackgerritAngus Lees proposed openstack-infra/system-config: Add libcap-dev in anticipation of python-prctl  https://review.openstack.org/16875000:35
*** davideagnello has joined #openstack-infra00:37
*** gokrokve has joined #openstack-infra00:39
*** davideagnello has quit IRC00:43
*** ddieterl_ has joined #openstack-infra00:44
*** gokrokve has quit IRC00:44
*** ddieterly has quit IRC00:44
*** smccully has quit IRC00:48
*** yamamoto has joined #openstack-infra00:49
*** jhesketh has quit IRC00:50
*** jhesketh has joined #openstack-infra00:53
*** tiswanso has joined #openstack-infra00:55
*** hdd has quit IRC00:59
*** imcsk8 has quit IRC01:00
*** imcsk8 has joined #openstack-infra01:00
*** achanda has joined #openstack-infra01:01
*** eharney has quit IRC01:04
*** tiswanso has quit IRC01:06
*** achanda has quit IRC01:06
*** tiswanso has joined #openstack-infra01:06
openstackgerritMatt Riedemann proposed openstack-infra/elastic-recheck: Add query for horizon integration test bug 1436903  https://review.openstack.org/16875301:07
openstackbug 1436903 in OpenStack Dashboard (Horizon) "integration tests failing blocking gate" [Critical,Triaged] https://launchpad.net/bugs/1436903 - Assigned to David Lyle (david-lyle)01:07
*** salv-orlando has joined #openstack-infra01:07
*** shakamunyi has joined #openstack-infra01:07
*** shakamunyi_ has joined #openstack-infra01:07
*** salv-orlando has quit IRC01:12
*** chris6131 has joined #openstack-infra01:14
*** Longgeek has joined #openstack-infra01:15
*** yamamoto has quit IRC01:15
*** ddieterly has joined #openstack-infra01:18
*** annegentle has quit IRC01:19
*** ddieterl_ has quit IRC01:21
*** masoz has quit IRC01:29
*** kaisers1 has joined #openstack-infra01:35
*** kaisers has quit IRC01:35
*** baoli has joined #openstack-infra01:37
*** gokrokve has joined #openstack-infra01:39
*** gokrokve has quit IRC01:44
*** stevemar has quit IRC01:45
*** stevemar has joined #openstack-infra01:46
*** yamamoto has joined #openstack-infra01:46
*** sdake has quit IRC01:51
*** ghostpl_ has joined #openstack-infra01:53
*** salv-orlando has joined #openstack-infra01:53
*** salv-orlando has quit IRC01:58
*** ghostpl_ has quit IRC02:00
*** baoli has quit IRC02:01
*** sputnik13 has quit IRC02:05
*** SumitNaiksatam has left #openstack-infra02:07
*** salv-orlando has joined #openstack-infra02:10
*** salv-orlando has quit IRC02:12
*** Longgeek has quit IRC02:12
*** SumitNaiksatam has joined #openstack-infra02:13
*** Longgeek has joined #openstack-infra02:16
*** Longgeek has quit IRC02:26
*** Longgeek has joined #openstack-infra02:26
openstackgerritAngus Lees proposed openstack/requirements: Add pyroute2 (python netlink bindings)  https://review.openstack.org/16876402:28
*** julim has quit IRC02:29
*** baoli has joined #openstack-infra02:29
*** Somay has quit IRC02:32
*** tiswanso has quit IRC02:32
*** mriedem1 has joined #openstack-infra02:37
*** mriedem has quit IRC02:39
*** gokrokve has joined #openstack-infra02:39
*** baoli has quit IRC02:42
*** gokrokve has quit IRC02:44
*** baoli has joined #openstack-infra02:45
*** annegentle has joined #openstack-infra02:45
*** sputnik13 has joined #openstack-infra02:51
*** annegentle has quit IRC02:51
*** shakamunyi__ has joined #openstack-infra02:51
*** david-ly_ is now known as david-lyle02:52
openstackgerritSteve Kowalik proposed openstack-infra/puppet-zanata: Add a zanata::client manifest  https://review.openstack.org/16708102:53
*** shakamunyi has quit IRC02:54
*** shakamunyi_ has quit IRC02:54
*** shakamunyi has joined #openstack-infra02:54
*** mriedem1 has quit IRC02:58
*** dims has quit IRC02:59
*** dims has joined #openstack-infra02:59
*** dims has quit IRC03:00
*** Longgeek_ has joined #openstack-infra03:07
*** Somay has joined #openstack-infra03:07
*** Longgeek has quit IRC03:10
*** jerryz_ has joined #openstack-infra03:10
*** sabeen has joined #openstack-infra03:11
*** ajmiller has quit IRC03:13
*** sabeen has quit IRC03:14
*** ayoung is now known as ayoung_ZZzz__03:21
*** medieval1_ has quit IRC03:24
*** armax has joined #openstack-infra03:24
*** salv-orlando has joined #openstack-infra03:25
*** armax has quit IRC03:25
*** ddieterly has quit IRC03:27
*** sandywalsh has quit IRC03:28
*** salv-orlando has quit IRC03:30
*** tkelsey has joined #openstack-infra03:36
*** sandywalsh has joined #openstack-infra03:37
*** otter768 has quit IRC03:38
*** gokrokve has joined #openstack-infra03:39
*** tkelsey has quit IRC03:41
*** hdd has joined #openstack-infra03:42
*** ghostpl_ has joined #openstack-infra03:43
*** gokrokve has quit IRC03:43
*** ghostpl_ has quit IRC03:48
*** baoli has quit IRC03:49
*** sdake has joined #openstack-infra03:49
*** sabeen1 has joined #openstack-infra03:58
*** sabeen1 has quit IRC04:05
*** dims has joined #openstack-infra04:15
*** woodster_ has quit IRC04:20
*** prmtl has joined #openstack-infra04:20
*** dims has quit IRC04:21
*** pal_ has joined #openstack-infra04:22
*** achanda has joined #openstack-infra04:24
*** ddieterly has joined #openstack-infra04:28
*** ddieterly has quit IRC04:32
*** shakamunyi_ has joined #openstack-infra04:39
*** gokrokve has joined #openstack-infra04:39
*** shakamunyi__ has quit IRC04:41
*** shakamunyi has quit IRC04:41
*** tnovacik has joined #openstack-infra04:41
*** shakamunyi has joined #openstack-infra04:41
*** gokrokve has quit IRC04:44
*** sdake has quit IRC04:49
*** pal__ has joined #openstack-infra04:53
*** pal_ has quit IRC04:56
*** BharatK has joined #openstack-infra04:57
*** masoz has joined #openstack-infra04:59
*** shakamunyi_ has quit IRC05:04
*** jerryz_ has quit IRC05:05
*** pal__ has quit IRC05:10
openstackgerritIan Wienand proposed openstack-infra/project-config: Restore devstack centos7 to non-voting check job  https://review.openstack.org/16878005:11
*** pal_ has joined #openstack-infra05:12
*** mrodden has quit IRC05:17
*** jerryz_ has joined #openstack-infra05:17
*** shakamunyi has quit IRC05:18
*** Alexandra_ has joined #openstack-infra05:19
*** unicell has quit IRC05:22
*** unicell has joined #openstack-infra05:22
*** achanda has quit IRC05:22
*** achanda_ has joined #openstack-infra05:22
*** shakamunyi has joined #openstack-infra05:22
*** jlibosva has joined #openstack-infra05:25
*** ildikov has quit IRC05:27
greghaynesjhesketh: Reviewing your zuul connection patch, having a hard time figuring out why https://review.openstack.org/#/c/121528/22/zuul/model.py is needed05:27
greghaynes(the none -> [] changes)05:27
greghaynesmakes me afraid were referencing those actions before the config is loaded, which seems like a possible race issue05:28
*** ddieterly has joined #openstack-infra05:29
*** achanda has joined #openstack-infra05:29
*** achanda_ has quit IRC05:30
greghaynesI removed it and tests still pass...05:30
*** chris6131 has quit IRC05:30
*** mrodden has joined #openstack-infra05:31
*** pal__ has joined #openstack-infra05:32
*** ghostpl_ has joined #openstack-infra05:32
*** ddieterly has quit IRC05:34
*** pal_ has quit IRC05:34
*** woodster_ has joined #openstack-infra05:37
*** ghostpl_ has quit IRC05:37
*** otter768 has joined #openstack-infra05:39
*** hdd has quit IRC05:39
*** otter768 has quit IRC05:43
*** tnovacik has quit IRC05:47
*** Somay has quit IRC05:48
jheskethgreghaynes: all the actions are looped over to send notifications... looping over an empty list is harmless and avoids the case where no merge_failure action types are set for example05:51
greghaynesjhesketh: Yes, but they should all be empty lists due to scheduler._parseConfig05:53
greghaynesas long as they are only being reference after the config is loaded, if they arent then for some reason were trying to perform actions before loading the config, which seems like it could be bad05:53
jheskethit's true, changing them to a list isn't needed here05:57
jheskethbut it denotes what is expected in those values05:57
jheskethhappy to change it back though05:57
greghaynesok, just making sure. I dont mind that they are I just wanted to make sure it wasnt a result of something bad05:57
*** prmtl has quit IRC06:01
*** pal_ has joined #openstack-infra06:02
*** pal__ has quit IRC06:05
greghaynesjhesketh: ok, replied, nothing huge but I think runtime imports are really important to avoid if at all possible06:07
greghaynesNeed to go to bed now, see you on the other side :)06:09
*** YorikSar has joined #openstack-infra06:10
*** prmtl has joined #openstack-infra06:10
*** e0ne has joined #openstack-infra06:19
*** jamielennox is now known as jamielennox|away06:20
*** deepakcs has joined #openstack-infra06:22
*** markvoelker has joined #openstack-infra06:22
*** e0ne has quit IRC06:23
*** shakamunyi has quit IRC06:25
*** yfried has joined #openstack-infra06:26
*** stevemar has quit IRC06:27
*** markvoelker has quit IRC06:27
*** stevemar has joined #openstack-infra06:27
*** e0ne has joined #openstack-infra06:28
openstackgerritJan Provaznik proposed openstack-infra/project-config: Create os-cloud-management project under TripleO  https://review.openstack.org/16543306:29
*** ddieterly has joined #openstack-infra06:29
*** ddieterly has quit IRC06:34
*** stevemar has quit IRC06:34
*** stevemar has joined #openstack-infra06:34
*** e0ne is now known as e0ne_06:35
*** e0ne_ is now known as e0ne06:35
*** e0ne has quit IRC06:36
*** Abhi_ has joined #openstack-infra06:42
Abhi_while running CI for any patch, does the change is merged automatically??06:44
Abhi_like we do the cherrypick??06:44
*** ildikov has joined #openstack-infra06:47
*** belmoreira has joined #openstack-infra06:48
*** tnovacik has joined #openstack-infra06:48
*** scheuran has joined #openstack-infra06:51
*** hichihara has quit IRC06:54
*** salv-orlando has joined #openstack-infra06:54
*** Ala has joined #openstack-infra06:57
*** salv-orl_ has joined #openstack-infra06:58
*** salv-orlando has quit IRC06:58
*** doude has joined #openstack-infra06:59
*** afazekas has joined #openstack-infra07:02
*** pal__ has joined #openstack-infra07:02
*** tkelsey has joined #openstack-infra07:04
*** pal_ has quit IRC07:05
*** sergsh has joined #openstack-infra07:07
*** achanda has quit IRC07:08
*** tkelsey has quit IRC07:08
*** mrda is now known as mrda-away07:12
*** ominakov has joined #openstack-infra07:13
*** doude_ has joined #openstack-infra07:18
*** stevemar has quit IRC07:19
*** ghostpl_ has joined #openstack-infra07:21
openstackgerrityolanda.robla proposed openstack-infra/storyboard-webclient: Initial proposal for integration tests  https://review.openstack.org/15621707:22
*** ihrachyshka has joined #openstack-infra07:22
*** doude has quit IRC07:22
*** markvoelker has joined #openstack-infra07:23
*** ominakov has quit IRC07:24
*** ibiris_away is now known as ibiris07:26
*** markvoelker has quit IRC07:27
*** jlibosva has quit IRC07:27
*** ghostpl_ has quit IRC07:28
deepakcsAbhi_, u mean whether the change is picked up automatically, then yes it is (I don't know whehter it does cherry pick or some other stuff tho')07:29
deepakcsAbhi_, but using 'merge' is the wrong way of putting it, since typically merge is done to master07:29
*** ildikov has quit IRC07:30
*** ddieterly has joined #openstack-infra07:30
deepakcsAbhi_, from a CI perspective, all you need to know is that the devstack job is setup with your patch included!07:30
Abhi_deepakcs, ok so it must be doing cherry -pick for a patch, and we are testing that cherry-pick through our CI, right??07:30
*** unicell has quit IRC07:30
*** Alexandra_ has quit IRC07:30
*** unicell has joined #openstack-infra07:31
deepakcsAbhi_, we are not testing cherry-pick, we are testing whether the change the patch introduced, is working fine with the vendor's storage backend using the CI infra07:31
*** shardy has joined #openstack-infra07:32
Abhi_ok thanks for the info07:33
*** ddieterly has quit IRC07:35
openstackgerrityolanda.robla proposed openstack-infra/storyboard-webclient: Initial proposal for integration tests  https://review.openstack.org/15621707:37
*** nmagnezi has joined #openstack-infra07:38
*** amotoki has joined #openstack-infra07:38
*** markus_z has joined #openstack-infra07:39
*** otter768 has joined #openstack-infra07:39
*** woodster_ has quit IRC07:40
*** arxcruz has joined #openstack-infra07:41
*** salv-orl_ has quit IRC07:42
openstackgerritJordan Pittier proposed openstack-infra/system-config: Third party CI: document how to properly format message with Gerrit Trigger  https://review.openstack.org/16771607:42
*** Somay has joined #openstack-infra07:42
*** jlanoux has joined #openstack-infra07:43
*** liusheng has quit IRC07:44
*** otter768 has quit IRC07:44
*** dulek has joined #openstack-infra07:44
*** ildikov has joined #openstack-infra07:45
*** jyuso1 has joined #openstack-infra07:46
*** cody-somerville has quit IRC07:47
*** hashar has joined #openstack-infra07:47
*** btully has quit IRC07:50
*** ominakov has joined #openstack-infra07:51
*** notnownikki has joined #openstack-infra07:51
*** ominakov has quit IRC07:52
*** mpaolino has joined #openstack-infra07:52
*** davideagnello has joined #openstack-infra07:53
*** dims has joined #openstack-infra07:53
*** tkelsey has joined #openstack-infra07:54
*** Somay has quit IRC07:54
*** pal_ has joined #openstack-infra07:55
*** yamamoto has quit IRC07:56
*** davideagnello has quit IRC07:57
*** pal__ has quit IRC07:57
*** dims has quit IRC07:58
*** tnovacik has quit IRC07:59
*** ssam2 has joined #openstack-infra08:00
openstackgerrityolanda.robla proposed openstack-infra/storyboard-webclient: Initial proposal for integration tests  https://review.openstack.org/15621708:02
*** cody-somerville has joined #openstack-infra08:03
*** HeOS has quit IRC08:04
*** vigneshvar has joined #openstack-infra08:05
*** vigneshvar_ has joined #openstack-infra08:05
*** _nadya_ has joined #openstack-infra08:05
*** tnovacik has joined #openstack-infra08:05
*** _nadya_ has quit IRC08:08
*** jistr has joined #openstack-infra08:09
*** jlibosva has joined #openstack-infra08:09
*** ominakov has joined #openstack-infra08:14
openstackgerrityolanda.robla proposed openstack-infra/storyboard-webclient: Initial proposal for integration tests  https://review.openstack.org/15621708:17
*** yamamoto has joined #openstack-infra08:19
*** ildikov has quit IRC08:23
*** pal__ has joined #openstack-infra08:25
*** yamamoto has quit IRC08:25
*** yfried is now known as yfried|afk08:25
*** pal_ has quit IRC08:28
*** ddieterly has joined #openstack-infra08:31
*** yfried|afk is now known as yfried08:31
*** MaxV has joined #openstack-infra08:33
*** kashyap has left #openstack-infra08:33
*** Abhi_ has quit IRC08:33
*** Somay has joined #openstack-infra08:34
*** teran has quit IRC08:34
*** ddieterly has quit IRC08:36
*** sergsh has quit IRC08:37
openstackgerrityolanda.robla proposed openstack-infra/storyboard: Add tools to run integration tests  https://review.openstack.org/15597508:38
*** ildikov has joined #openstack-infra08:38
*** ghostpl_ has joined #openstack-infra08:38
*** ihrachyshka has quit IRC08:39
*** Ala has quit IRC08:39
*** dtantsur|afk is now known as dtantsur08:40
*** salv-orlando has joined #openstack-infra08:43
*** e0ne has joined #openstack-infra08:43
*** btully has joined #openstack-infra08:46
*** ghostpl_ has quit IRC08:47
*** salv-orlando has quit IRC08:47
*** dhritishikhar_ has joined #openstack-infra08:48
*** sergsh has joined #openstack-infra08:49
*** btully has quit IRC08:50
*** dhritishikhar__ has joined #openstack-infra08:51
*** e0ne is now known as e0ne_08:51
*** dhritishikhar_ has quit IRC08:51
*** Longgeek_ has quit IRC08:53
*** e0ne_ is now known as e0ne08:54
*** vigneshvar has quit IRC08:56
*** ZZelle has joined #openstack-infra08:56
*** vigneshvar_ has quit IRC08:56
*** jlibosva has quit IRC08:58
*** jlibosva has joined #openstack-infra09:02
*** jlibosva has quit IRC09:06
*** ominakov has quit IRC09:07
*** zz_johnthetubagu is now known as johnthetubaguy09:07
*** jlibosva has joined #openstack-infra09:09
*** jlibosva has quit IRC09:10
*** jlibosva has joined #openstack-infra09:11
*** ominakov has joined #openstack-infra09:12
*** HeOS has joined #openstack-infra09:17
*** ociuhandu has joined #openstack-infra09:17
*** dhritishikhar__ has quit IRC09:19
*** pal_ has joined #openstack-infra09:21
*** ihrachyshka has joined #openstack-infra09:21
*** pal__ has quit IRC09:23
*** markvoelker has joined #openstack-infra09:24
*** ajo has joined #openstack-infra09:25
*** mpaolino has quit IRC09:25
*** smccully has joined #openstack-infra09:26
*** ghostpl_ has joined #openstack-infra09:27
*** markvoelker has quit IRC09:29
*** ajo_ has joined #openstack-infra09:30
*** ddieterly has joined #openstack-infra09:32
*** jlibosva has quit IRC09:33
*** ajo has quit IRC09:33
*** flip214 has joined #openstack-infra09:36
*** ddieterly has quit IRC09:36
*** derekh has joined #openstack-infra09:36
flip214clarkb: ping09:37
*** Somay has quit IRC09:37
*** dtantsur is now known as dtantsur|brb09:37
*** dizquierdo has joined #openstack-infra09:38
*** ajo_ is now known as ajo09:38
*** ajo has quit IRC09:40
*** ajo has joined #openstack-infra09:40
*** otter768 has joined #openstack-infra09:40
openstackgerritAleksey Ripinen proposed openstack-infra/storyboard: Branches, milestones and projects ids validation in tasks  https://review.openstack.org/16212609:41
*** otter768 has quit IRC09:45
*** skolekonov_ has joined #openstack-infra09:46
*** rlandy has joined #openstack-infra09:49
*** teran has joined #openstack-infra09:50
*** pal__ has joined #openstack-infra09:53
*** dims has joined #openstack-infra09:54
*** teran_ has joined #openstack-infra09:54
*** pal_ has quit IRC09:56
*** teran has quit IRC09:57
*** vigneshvar has joined #openstack-infra09:58
*** Longgeek has joined #openstack-infra09:58
*** vigneshvar_ has joined #openstack-infra09:59
*** ominakov has quit IRC10:01
openstackgerritMerged openstack-infra/storyboard: Refactor Tasks to work as a Subcontroller  https://review.openstack.org/15005210:02
*** ominakov has joined #openstack-infra10:02
*** ominakov has quit IRC10:03
openstackgerritMerged openstack-infra/storyboard: Added events for tags  https://review.openstack.org/16507410:03
*** ssam2 has quit IRC10:06
openstackgerritMerged openstack-infra/storyboard: Removed Nickname  https://review.openstack.org/16519410:07
kaisers1Hi! Is anybody around for a devstack config question?10:10
*** yamamoto has joined #openstack-infra10:12
*** flip214 has quit IRC10:17
*** flip214 has joined #openstack-infra10:17
openstackgerritAlex proposed openstack-infra/subunit2sql: Extend database schema of run_metadata  https://review.openstack.org/16884010:17
*** Longgeek has quit IRC10:17
*** ssam2 has joined #openstack-infra10:20
*** _nadya_ has joined #openstack-infra10:20
*** samueldmq-away is now known as samueldmq10:22
*** pal_ has joined #openstack-infra10:23
*** salv-orlando has joined #openstack-infra10:23
*** markvoelker has joined #openstack-infra10:25
*** jlibosva has joined #openstack-infra10:26
*** pal__ has quit IRC10:26
*** markvoelker has quit IRC10:30
*** pblaho has joined #openstack-infra10:32
*** yfried is now known as yfried|afk10:32
*** ddieterly has joined #openstack-infra10:32
*** tnovacik has quit IRC10:33
*** hashar has quit IRC10:33
*** hashar has joined #openstack-infra10:34
*** yfried|afk is now known as yfried10:34
*** yamamoto has quit IRC10:35
*** pc_m has joined #openstack-infra10:36
*** ddieterly has quit IRC10:37
*** ominakov has joined #openstack-infra10:40
*** johnthetubaguy is now known as zz_johnthetubagu10:41
*** prmtl has quit IRC10:41
*** prmtl has joined #openstack-infra10:42
*** yamamoto has joined #openstack-infra10:42
*** mpavone has joined #openstack-infra10:48
*** dtantsur|brb is now known as dtantsur10:50
*** pal_ has quit IRC10:52
*** mwagner_lap has quit IRC10:52
*** yamamoto has quit IRC10:52
*** pal_ has joined #openstack-infra10:52
openstackgerritAleksey Ripinen proposed openstack-infra/storyboard: Branches, milestones and projects ids validation in tasks  https://review.openstack.org/16212610:55
*** tnovacik has joined #openstack-infra10:57
*** pal_ has quit IRC11:01
*** ominakov has quit IRC11:02
*** ominakov has joined #openstack-infra11:03
*** vigneshvar_ has quit IRC11:04
*** vigneshvar has quit IRC11:04
*** vigneshvar has joined #openstack-infra11:05
*** vigneshvar_ has joined #openstack-infra11:05
*** ominakov has quit IRC11:06
*** marcusvrn has joined #openstack-infra11:08
*** marcusvrn1 has joined #openstack-infra11:08
*** ominakov has joined #openstack-infra11:11
*** sergsh has quit IRC11:11
*** marcusvrn has quit IRC11:13
openstackgerritAleksey Ripinen proposed openstack-infra/storyboard: Added story types  https://review.openstack.org/16341211:15
*** Longgeek has joined #openstack-infra11:15
*** ominakov has quit IRC11:15
*** jaypipes has joined #openstack-infra11:17
*** prmtl has quit IRC11:18
flip214hi, short questions about http://lists.openstack.org/pipermail/openstack-dev/2015-February/057585.html.11:21
flip214in this ML post clarkb had two gerrit links, which I'm not sure whether they're the correct ones.11:21
flip214does gerrit reuse its review numbers?11:22
*** AJaeger has joined #openstack-infra11:22
*** AJaeger has joined #openstack-infra11:22
*** dhritishikhar_ has joined #openstack-infra11:23
openstackgerritAleksey Ripinen proposed openstack-infra/storyboard: Branches, milestones and projects ids validation in tasks  https://review.openstack.org/16212611:25
*** lucasagomes has joined #openstack-infra11:25
*** sergsh has joined #openstack-infra11:26
*** markvoelker has joined #openstack-infra11:26
lucasagomeshi, if you have a time please take a look at https://review.openstack.org/#/c/163344/ , it add a non-voting check on the openstack/ironic-python-agent project to test IPA with the PXE ramdisks. It has been approved once but it had to be rebased. (Need one more +2/+A now). Tanks11:26
lucasagomesthanks*11:26
*** markus_z has quit IRC11:27
bookwarhi, we have hanging merge here https://review.openstack.org/#/c/167976/ and i don't see it in zuul, how can this be resolved?11:28
*** lucasagomes is now known as lucas-hungry11:28
openstackgerritAleksey Ripinen proposed openstack-infra/storyboard: Branches, milestones and projects ids validation in tasks  https://review.openstack.org/16212611:29
*** markvoelker has quit IRC11:31
openstackgerritAleksey Ripinen proposed openstack-infra/storyboard: Added story types  https://review.openstack.org/16341211:32
*** BharatK has quit IRC11:33
*** miqui has quit IRC11:33
*** ddieterly has joined #openstack-infra11:33
*** btully has joined #openstack-infra11:35
*** doug-fish has joined #openstack-infra11:36
hughsaundersHow can I filter by subject/commit message in gerrit? I want to exclude WIPs (some commits are marked WIP in the subject, not in the workflow field)11:37
*** Murad has joined #openstack-infra11:37
*** ddieterly has quit IRC11:38
Muradcan anybody help in creating html report for tempest?11:39
*** btully has quit IRC11:40
*** Longgeek has quit IRC11:40
*** otter768 has joined #openstack-infra11:41
*** ldnunes has joined #openstack-infra11:42
*** otter768 has quit IRC11:46
openstackgerritAleksey Ripinen proposed openstack-infra/storyboard: Added request statistic  https://review.openstack.org/16469611:46
openstackgerritAleksey Ripinen proposed openstack-infra/storyboard-webclient: Support for tags  https://review.openstack.org/16684711:48
*** deepakcs has quit IRC11:49
*** vigneshvar has quit IRC11:49
*** vigneshvar_ has quit IRC11:50
*** yamamoto has joined #openstack-infra11:50
*** kgiusti has joined #openstack-infra11:51
*** dhritishikhar_ has quit IRC11:58
*** woodster_ has joined #openstack-infra12:00
*** dulek_ has joined #openstack-infra12:03
*** ominakov has joined #openstack-infra12:03
*** dulek has quit IRC12:06
*** jistr has quit IRC12:07
*** ominakov has quit IRC12:07
*** jistr has joined #openstack-infra12:08
*** cdent has joined #openstack-infra12:08
*** markvoelker has joined #openstack-infra12:10
*** dulek___ has joined #openstack-infra12:11
*** dulek_ has quit IRC12:14
*** prmtl has joined #openstack-infra12:14
*** prmtl has joined #openstack-infra12:14
*** ildikov has quit IRC12:17
*** dims has quit IRC12:17
*** dims has joined #openstack-infra12:18
*** dkranz has joined #openstack-infra12:18
*** krtaylor has quit IRC12:19
*** yfried is now known as yfried|afk12:20
*** Ala has joined #openstack-infra12:21
*** bswartz has quit IRC12:22
*** dulek___ is now known as dulek12:23
*** aysyd has joined #openstack-infra12:23
*** salv-orlando has quit IRC12:27
*** dkliban_afk is now known as dkliban12:27
*** gordc has joined #openstack-infra12:28
*** ayoung_ZZzz__ is now known as ayoung_short12:28
*** ayoung_short is now known as ayoung_cough12:28
*** ayoung_cough is now known as ayoung_yawn12:29
*** ayoung_yawn is now known as ayoung12:29
*** salv-orlando has joined #openstack-infra12:30
*** sputnik13 has quit IRC12:30
*** weshay has joined #openstack-infra12:30
*** smccully has quit IRC12:31
*** yfried|afk is now known as yfried12:31
*** pal_ has joined #openstack-infra12:32
*** rfolco has joined #openstack-infra12:32
*** lucas-hungry is now known as lucasagomes12:32
openstackgerritMerged openstack-infra/project-config: Ironic: Create and enable pxe_ipa-src-nv on ironic-python-agent  https://review.openstack.org/16334412:34
*** ddieterly has joined #openstack-infra12:34
openstackgerritAndreas Jaeger proposed openstack-infra/project-config: Move gantt,python-gantclient to attic  https://review.openstack.org/16888312:35
*** Longgeek has joined #openstack-infra12:36
*** sandywalsh has quit IRC12:37
*** tiswanso has joined #openstack-infra12:38
*** ddieterly has quit IRC12:39
*** sandywalsh has joined #openstack-infra12:39
openstackgerritAleksey Ripinen proposed openstack-infra/storyboard: Added request statistic  https://review.openstack.org/16469612:40
*** baoli has joined #openstack-infra12:42
*** ildikov has joined #openstack-infra12:43
*** ddieterly has joined #openstack-infra12:44
*** e0ne is now known as e0ne_12:44
*** e0ne_ is now known as e0ne12:51
*** lascii is now known as alaski12:52
*** timcline has joined #openstack-infra12:52
*** timcline has quit IRC12:52
*** dkranz has quit IRC12:53
*** mwagner_lap has joined #openstack-infra12:53
*** timcline has joined #openstack-infra12:53
*** tnovacik has quit IRC12:54
*** dboik has quit IRC12:56
*** ddieterly has quit IRC12:56
*** shakamunyi has joined #openstack-infra12:57
openstackgerritAndreas Jaeger proposed openstack-infra/project-config: Move gantt,python-gantclient to attic  https://review.openstack.org/16888312:58
*** timcline has quit IRC12:59
*** julim has joined #openstack-infra12:59
*** krtaylor has joined #openstack-infra13:00
*** julim has quit IRC13:01
anteayaflip214: gerrit doesn't reuse its review numbers13:01
*** miqui has joined #openstack-infra13:02
flip214anteaya: thank you for your answer!13:03
flip214the link in the ML post https://review.openstack.org/#/c/141530/ seems to have different contents, though...13:03
*** e0ne is now known as e0ne_13:03
flip214I remember that the gerrit data matched the textual description "current state of multinode work" some weeks ago,13:04
flip214but now it's title is "Set custom cpu_model for live_migrate"13:04
anteayabookwar: I don't see any Jenkins reports on that patch so I don't know what it is doing13:04
*** dkranz has joined #openstack-infra13:04
*** bswartz has joined #openstack-infra13:05
anteayahughsaunders: if you use label:Workflow=0 that removes all wip patches13:05
*** bknudson has quit IRC13:05
anteayaand also any approved patches13:05
hughsaundersanteaya: problem is that some people use "WIP" in the subject rather than the workflow column13:06
*** hrubi_ has quit IRC13:06
*** lucasagomes has left #openstack-infra13:06
anteayayou can additionally use label:Verified=+1,jenkins to filter on patches that don't have +1 returned from Jenkins, so either patches currently running check tests or patches that have failed check13:06
bookwaranteaya: the jenkins is configured to do noop check and the merge itself, here is for example it works: https://review.openstack.org/#/c/167662/ but the latest patch didn't trigger Jenkins task13:06
hughsaundersanteaya: is it possible to filter on commit message?13:07
anteayahughsaunders: I have no ability to filter out people who don't use gerrit at it is intended to be used13:07
anteayaflip214: I confirm your findings13:07
*** Hal has joined #openstack-infra13:07
anteayabookwar: yes I confirm what you are seeing13:08
*** Hal has quit IRC13:08
hughsaundersanteaya: gerrit does have a message: query, but it doesn't seem to match the commit message, is there another way to do that?13:08
*** annegentle has joined #openstack-infra13:08
bookwari'll ask devs to reset Workflow and CR mark one's again, maybe it'll help13:09
*** Longgeek has quit IRC13:10
*** Hal has joined #openstack-infra13:11
anteayahughsaunders: I personally haven't had good luck with the message query13:11
*** Hal is now known as Guest3032413:11
boris-42anteaya: hi13:11
anteayahughsaunders: I have found that for me it returns all sorts of results with have nothing to do with with my query13:12
hughsaundersanteaya: I have also experienced that13:12
*** julim has joined #openstack-infra13:12
anteayahughsaunders: I also have been unsuccessful in getting it to take more than one word for the query13:12
anteayahughsaunders: which is why I say that I have no ability to filter for people who don't use gerrit as it was intended13:12
*** shakamunyi has quit IRC13:13
anteayabookwar: it might13:13
hughsaundersanteaya: I wonder if thats a gerrit upstream issue or our implementation of gerrit?13:13
anteayahughsaunders: well our implementation of gerrit draws from upstream13:13
anteayawe haven't done any customization of the gerrit query functionality13:14
anteayato the best of my knowledge13:14
anteayawhich is why I stick to the labels13:14
*** e0ne_ has quit IRC13:14
hughsaundersis it possible to get workflow -1 to stick across patchsets?13:15
*** esker has quit IRC13:15
anteayaboris-42: how about you just post to the channel, my average of questions I can answer compared to questions asked today is not very high13:15
*** ghostpl_ has quit IRC13:15
anteayaboris-42: I have no confidence that I can help you boris-4213:15
*** dboik has joined #openstack-infra13:16
anteayahughsaunders: well, the only vote that sticks across patchsets is a -2 from a reviewer13:16
*** e0ne has joined #openstack-infra13:16
anteayahughsaunders: since having workflow -1 stick across patchsets interferes with the functionality of workflow -113:16
openstackgerritMehdi Abaakouk proposed openstack-infra/project-config: Update gnocchi repository url  https://review.openstack.org/16889513:17
anteayahughsaunders: you seem to be dealing with something rather specific but asking general questions to get there13:17
anteayahughsaunders: are you going to tell me what you are actually dealing with?13:17
hughsaundersanteaya: XY13:17
anteayawhat does that mean?13:17
*** dprince has joined #openstack-infra13:18
hughsaundersanteaya: because workflow -1 doesn't stick across patchsets, reviews get labelled with WIP in the subject, rather than using the workflow field. Which means I can't filter them :(13:18
anteayaright13:18
hughsaundersxy problem: http://xyproblem.info/13:18
anteayabecause that is developer behaviour13:18
silehtHi, does anyone can review https://review.openstack.org/#/c/168895/, it blocks the gate for gnocchi since we have moved the repository to the openstack namespace.13:19
anteayahughsaunders: I suggest you bring it up at the weekly project meeting of whatever project you are working on13:19
anteayahughsaunders: and present your perspective13:20
anteayapersonally I just reset workflow -1 on a wip patch13:20
anteayait isn't that hard13:20
*** peristeri has joined #openstack-infra13:20
anteayaand yes, the link describes it beautifully, thank you13:21
*** zz_johnthetubagu is now known as johnthetubaguy13:21
hughsaundersanteaya: I'll check if git-review has a flag for w-1, maybe a useful feature if not there already13:21
anteayasileht: hi, hopefully you can get another +2 before too long13:22
anteayahughsaunders: no argument from me13:22
silehtanteaya, thanks13:22
anteayasileht: welcome, sorry I didn't catch that in review13:22
openstackgerritMehdi Abaakouk proposed openstack-infra/project-config: Update gnocchi repository url  https://review.openstack.org/16889513:23
flip214anteaya: well, so what can I do about that? Is #infra interested in taking a look what happened there?13:23
silehtanteaya, I have updated the review (I have missed 1 occurence of stackforge)13:24
*** dustins has joined #openstack-infra13:24
anteayaflip214: I can only speak for myself, you asked a few questions, I was able to answer one of them and answered the one I could13:25
anteayaflip214: I can't give you information I don't have13:25
*** sdake has joined #openstack-infra13:25
anteayasileht: I will review again13:25
silehtanteaya, thanks13:25
flip214anteaya: well, who can I ask about gerrit issues like these?13:27
*** sdake_ has joined #openstack-infra13:28
anteayasileht: okay I grepped this time and found a total of two entries that need to be changed, looks like you found them both13:28
*** tnovacik has joined #openstack-infra13:28
*** hdd has joined #openstack-infra13:29
*** ssam2 has quit IRC13:30
*** sdake has quit IRC13:31
AJaegeranteaya, Good morning! Could you review https://review.openstack.org/#/c/168022/ , please?13:31
flip214anteaya: thanks a lot, btw!13:32
openstackgerritCyril Roelandt proposed openstack-infra/project-config: saharaclient: enable Python3 jobs  https://review.openstack.org/16889913:33
*** johnthetubaguy is now known as zz_johnthetubagu13:34
anteayaAJaeger: done13:35
anteayaflip214: you're welcome13:35
AJaegerthanks, anteaya13:35
anteayaAJaeger: thanks for reviewing sileht's patch13:36
AJaegeranteaya, sure13:36
*** chlong has joined #openstack-infra13:36
silehtAJaeger, thx13:36
* anteaya goes to find some breakfast13:36
AJaegeranteaya, enjoy!13:37
*** ddieterly has joined #openstack-infra13:37
*** hdd has quit IRC13:38
*** deepakcs has joined #openstack-infra13:39
*** rfolco has quit IRC13:40
*** jlanoux has quit IRC13:40
*** eharney has joined #openstack-infra13:41
*** otter768 has joined #openstack-infra13:42
*** rfolco has joined #openstack-infra13:42
*** mriedem has joined #openstack-infra13:42
*** markus_z has joined #openstack-infra13:43
*** e0ne is now known as e0ne_13:43
*** ssam2 has joined #openstack-infra13:44
fungihughsaunders: workflow -1 doesn't persist on new patchsets because not persisting on new patchsets was a desired behavior of workflow -113:44
*** jgrimm is now known as zz_jgrimm13:45
*** bknudson has joined #openstack-infra13:45
fungihughsaunders: it's currently implemented as an individual vote, not a change state, so the only ways to clear it are either to reverse your own workflow -1 vote or to upload a new patchset to a change. if workflow -1 persisted on new patchsets then the only way to remove a workflow -1 would be to undo the vote yourself. not great if you're collaborating on a change with the rest of your team13:46
openstackgerritMerged openstack-infra/project-config: Enable PyPi push for the stackforge/faafo project  https://review.openstack.org/16802213:46
*** e0ne_ is now known as e0ne13:46
hughsaundersfungi: thanks, I hadn't twigged that people could disagree on the workflow state.13:46
openstackgerritMerged openstack-infra/project-config: Update gnocchi repository url  https://review.openstack.org/16889513:47
*** rfolco has quit IRC13:47
*** otter768 has quit IRC13:47
*** rfolco has joined #openstack-infra13:48
*** claudiupopa has joined #openstack-infra13:49
*** sdake has joined #openstack-infra13:49
jogoclarkb: oh you got 64 bit live migration working, nice13:50
*** ghostpl_ has joined #openstack-infra13:51
*** stevemar has joined #openstack-infra13:52
*** sdake_ has quit IRC13:53
*** tonytan4ever has joined #openstack-infra13:53
*** xyang1 has joined #openstack-infra13:53
openstackgerritMerged openstack-infra/project-config: Add new project nerd-reviewer  https://review.openstack.org/16694513:53
openstackgerritMerged openstack-infra/project-config: Add new project nerd-reviewer  https://review.openstack.org/16694513:53
*** alexpilotti has joined #openstack-infra13:53
*** spzala has joined #openstack-infra13:54
*** superdan is now known as dansmith13:56
*** sabeen1 has joined #openstack-infra13:56
fungihughsaunders: when we upgrade to gerrit 2.9 i think we'll have the ability to start using the wip plugin which has been under development for our use case. it implements a custom change state rather than using a vote, and so can be switched on and off by different users13:57
hughsaundersfungi: sounds good13:58
*** baoli has quit IRC13:59
*** baoli_ has joined #openstack-infra14:00
*** dtantsur is now known as dtantsur|brb14:01
*** pal_ has quit IRC14:01
*** gokrokve has joined #openstack-infra14:02
*** yamamoto has quit IRC14:03
*** yamamoto has joined #openstack-infra14:03
*** dboik_ has joined #openstack-infra14:03
fungiflip214: you'd probably have to ask clarkb what he meant when linking https://review.openstack.org/141530 in http://lists.openstack.org/pipermail/openstack-dev/2015-February/057585.html14:04
fungiflip214: as far as i can tell the change has always had that title all the way back to december when it was first uploaded14:05
*** ominakov has joined #openstack-infra14:05
*** jyuso has joined #openstack-infra14:05
*** jyuso1 has quit IRC14:05
fungiflip214: it's possible clarkb meant to link a different change14:05
*** dboik has quit IRC14:06
*** nelsnelson has joined #openstack-infra14:09
*** doude_ has quit IRC14:09
*** doude has joined #openstack-infra14:09
*** sigmavirus24_awa is now known as sigmavirus2414:09
*** yamamoto has quit IRC14:10
*** tkelsey has quit IRC14:10
*** btully has joined #openstack-infra14:11
*** dkranz has quit IRC14:11
*** ominakov has quit IRC14:12
fboHi, I saw there is lot of errors in the system-config integration test "apply-test.sh" logs run by the gate. So I'm wondering if is it still relevant ?14:12
anteayafbo: have you a link to the errors you are seeing?14:13
fboanteaya, yep http://logs.openstack.org/52/166252/4/check/gate-infra-puppet-apply-precise/be3a209/console.html14:14
fboanteaya, I see this kind of error "Skipping because of failed dependencies" for all patches and for all tests on precise/trusty/centos6 platform14:15
anteayaokay looking at the tests for this patch, the apply tests failed one time and all three apply tests failed14:16
fboBut it seems the script reports a success to gerrit14:16
anteayaevery other test run on that patch had passing apply tests14:16
fungifbo: right now it's not safe/easy to do a full apply of all our manifests together on one machine, so we use --noop14:17
fungifbo: unfortunately since there are many operations puppet won't perform in --noop mode it sees a lot of dependent actions as unsatisfied, hence the error messages14:17
fbofungi, yes I agree not saw easy I guess14:17
*** jcoufal has joined #openstack-infra14:18
*** tonytan_brb has joined #openstack-infra14:18
*** mtanino has joined #openstack-infra14:18
anteayafbo: so based on my initial findings so far I'm going with yes it is still relevant14:19
fbofungi, and do you have some ideas to improve the integration testing for the system-config manifests ?14:20
*** bhunter71 has joined #openstack-infra14:20
*** timcline has joined #openstack-infra14:20
kaisers1anteaya: Hi! is there a thirdparty mentoring today?14:21
*** tonytan4ever has quit IRC14:22
fungifbo: probably best to talk to nibalizer since i think he was working on something to address that14:22
*** mattfarina has joined #openstack-infra14:23
anteayakaisers1: there is a third-party meeting today14:24
fboanteaya, ok because I sent some patches last week on system-config but I was a bit lost with all that log output and wasn't able to check if my patches pass the integration test or not :)14:24
anteayakaisers1: I have no idea why you call it mentoring14:25
anteayafbo: did jenkins report success or failure?14:25
kaisers1anteaya: sorry, i seemed to remember it was called that.14:26
*** jlanoux has joined #openstack-infra14:26
*** dkranz has joined #openstack-infra14:27
fbofungi, Ok I'll do that, I've some experience with this kind of complex testing using LXC containers and I'll be happy to help if I can :)14:27
fboanteaya, yes jenkins reports a success but it should not as manifest apply fail14:28
*** ghostpl_ has quit IRC14:28
*** ajo has quit IRC14:28
*** ominakov has joined #openstack-infra14:29
*** ghostpl_ has joined #openstack-infra14:29
*** garyh has joined #openstack-infra14:29
anteayakaisers1: I never called it that14:29
fungifbo: puppet "failures" aren't black and white. it returns a variety of different exit codes for different conditions, so we catch specific ones we know indicate a problem14:29
*** ChuckC has quit IRC14:29
fungidboik_: it's mostly there to catch things like parsing errors14:30
fungier, fbo ^14:30
fungi(sorrt dboik_)14:30
*** jlanoux_ has joined #openstack-infra14:31
kaisers1anteaya: apologies, it was called like that in some cinder CI related mails, that's where my impression came from. Just looked it up. I won't address it like that no more! :)14:31
*** wenlock has joined #openstack-infra14:32
*** achanda has joined #openstack-infra14:33
*** jlanoux has quit IRC14:34
anteayakaisers1: thanks14:34
anteayaI'll talk to cinder folks14:34
*** ghostpl_ has quit IRC14:34
anteayaand yes 1500 is meeting time14:35
*** ghostpl_ has joined #openstack-infra14:35
*** prmtl_ has joined #openstack-infra14:36
*** tonytan_brb is now known as tonytan4ever14:36
*** jlibosva has quit IRC14:37
*** ghostpl_ has quit IRC14:37
*** asselin has quit IRC14:37
*** tkelsey has joined #openstack-infra14:37
*** asselin has joined #openstack-infra14:37
*** ildikov has quit IRC14:38
*** ilyashakhat has joined #openstack-infra14:38
*** ghostpl_ has joined #openstack-infra14:38
*** zz_dimtruck is now known as dimtruck14:38
*** prmtl has quit IRC14:40
claudiupopamordred: ping14:40
*** asselin has quit IRC14:40
*** asselin has joined #openstack-infra14:41
*** ghostpl_ has quit IRC14:41
*** asselin has quit IRC14:41
*** tkelsey has quit IRC14:41
*** asselin_ has joined #openstack-infra14:41
*** asselin_ has quit IRC14:42
*** patrickeast has joined #openstack-infra14:42
*** asselin_ has joined #openstack-infra14:43
*** ghostpl_ has joined #openstack-infra14:43
*** asselin_ has quit IRC14:44
*** achanda has quit IRC14:44
*** asselin_ has joined #openstack-infra14:44
*** zz_jgrimm is now known as jgrimm14:44
*** asselin_ has quit IRC14:45
*** asselin_ has joined #openstack-infra14:45
*** ominakov has quit IRC14:46
*** asselin_ has quit IRC14:46
*** asselin has joined #openstack-infra14:47
*** asselin has quit IRC14:47
*** zul has quit IRC14:47
*** asselin has joined #openstack-infra14:48
*** nmagnezi has quit IRC14:48
*** yfried has quit IRC14:48
*** asselin has quit IRC14:49
*** asselin_ has joined #openstack-infra14:49
*** armax has joined #openstack-infra14:49
*** timcline has quit IRC14:49
mordredclaudiupopa: hi14:50
*** asselin_ has quit IRC14:50
*** asselin_ has joined #openstack-infra14:50
*** asselin_ has quit IRC14:51
*** thingee has quit IRC14:51
*** asselin_ has joined #openstack-infra14:51
*** skolekonov_ has quit IRC14:51
*** dtantsur|brb is now known as dtantsur14:52
clarkbfungi flip214 thats where multinode work was. setting cpu models to enable nova live migration14:52
*** asselin_ has quit IRC14:52
*** asselin has joined #openstack-infra14:52
*** asselin_ has joined #openstack-infra14:52
fungiclarkb: which seems more like something you're testing on multinode (live migration) rather than something necessary to make multinode work (which seemed more like the point of the ml reply) so that's probably the confusion14:53
*** zul has joined #openstack-infra14:53
clarkbmaybe, it is about workong multinode because libvirt14:53
*** MarkAtwood has joined #openstack-infra14:53
fungie.g. are cpu model settings relevant to testing drbd backend?14:54
claudiupopamordred: any idea why this change isn't in pbr 0.10? https://review.openstack.org/#/c/136707/14:54
clarkbfungi I have no idea because I dont know what specialsauce they are doing14:54
fungii guess multi-node devstack is broken without the cpu settings?14:54
clarkbfungi yes14:54
claudiupopait's non on pypi either.14:54
mordredclaudiupopa: that's right14:54
mordredclaudiupopa: currently master is not releasable because there is a bug in the semver support there14:55
mordredso bugfixes have been going out to the 0.10 when necessary on a stable/0.10 branch which we're not particularly happy about14:55
*** Longgeek has joined #openstack-infra14:55
mordredhowever, I think we're pretty fed up with waiting for the semver stuff to be fixed14:55
fungiclaudiupopa: master in pbr is essentially a development branch right now, and the releases have been coming from a separate "feature/0.10" release branch lately14:55
*** thedodd has joined #openstack-infra14:56
claudiupopaOh I see.14:56
fboThanks fungi for the explanation I understand this cannot be really called integration tests for now.14:56
yolandahi mordred, wanted to share with you that change :https://review.openstack.org/#/c/168686/14:56
yolandawhat do you think about it?14:56
clarkbfungi anyways I was pointing at general work for multinode. It had nothing to do directly with drbd other than indicating where to go for multinode stuff14:56
*** tkelsey has joined #openstack-infra14:57
clarkbI encouraged them to start witha single node test which has not happened14:57
mordredfungi: perhaps we should send a message to the the list that if someone doesn't step up to finish the semver patches, we're going to consider them abandoned, move them to a feature branch and re-promote 0.10 to master14:57
*** timcline has joined #openstack-infra14:57
mordredfungi: with a timeframe - maybe a week?14:57
claudiupopamordred: Well, couldnt this change be cherrypicked? ;-) We were believing it's there already and we just had one bad release on cloudbaseinit because to it.14:57
*** Guest96413 is now known as redrobot14:58
mordredclaudiupopa: you are more than welcome to propose a cherry-pick of the patch to feature/0.10! :)14:58
jogofungi: question related to what to do with gantt14:58
jogofungi: what is the stance on changing the history of a repo (equivalent of git push --force)?14:58
jogofungi: because that is what will happen if we re-use the existing gantt repo14:59
anteayajogo: why?14:59
jeblairjogo: we have two choices -- we could do what we did with keystone and replace it while keeping history (basically, new history into a branch and then merge that branch into master)15:00
*** timcline_ has joined #openstack-infra15:00
jeblairjogo: or we could rename gantt to something else and rename the new thing to gantt15:00
jeblairjogo: it kind of depends on the degree of relation between the two15:00
*** reed has joined #openstack-infra15:01
*** rwsu has joined #openstack-infra15:01
jeblairjogo: but we will never erase history with "push --force"  *15:01
jogojeblair: the contents of the existing gantt repo are worthless and confusing15:01
jeblair* (except for really minor things like an incorrect commit on a repo import or something technical like that)15:01
mordredjeblair: ++15:01
jeblairjogo: that's not a value judgement any of us get to make :)15:01
jogoand there will be no relationship between the existing content and what the new scheduler split would contain15:02
fungijogo: might be a good idea to merge a commit which deletes all the files and replaces them with an explanatory readme about the situation if you're concerned about it being confusing in its current state?15:02
jeblairjogo: that is a more compelling argument to do a rename swap :)15:02
jogohttps://github.com/openstack/gantt#disclaimer15:02
*** zul has quit IRC15:02
fungii see it as a very compelling argument to use a new name for the project15:02
jogojeblair: rename swap?15:02
*** zul has joined #openstack-infra15:03
mordredjogo: 15:00:01          jeblair | jogo: we have two choices -- we could do what we did with keystone and replace it while keeping history (basically, new history into a15:03
mordred                          | branch and then merge that branch into master)15:03
jogofungi: ++, but I will defer to jaypipes on what makes the most sense15:03
*** timcline has quit IRC15:03
jogoahh15:03
jeblairfungi: ah, yes, the zeroth option.  i did forget to mention that.  it's a good one.  :)15:04
mordredjogo: we're never going to delete git history, but we can make the repo look like what whatever people want15:04
*** dboik_ has quit IRC15:04
jogomordred: ack15:04
jeblairjogo: are you going to pull code out of nova and put it into gantt?15:04
jeblair* scheduler code15:04
jogojeblair: honestly don't actually know at this point15:04
*** dboik has joined #openstack-infra15:04
jogojeblair: that was the idea, but not sure if that is still the idea. Once again I defer to jaypipes15:05
*** SumitNaiksatam has quit IRC15:05
mordredyolanda: that's an interesting idea ... it makes https://review.openstack.org/#/c/168665/ a bit different to reason about15:05
*** SumitNaiksatam has joined #openstack-infra15:05
*** arxcruz has quit IRC15:05
mordredyolanda: in terms of where a priority would be expressed, because I don't think any of those actions would have inherent priorities that would make sense to include in an API15:06
anteayajogo: if I can suggest, why don't you and jaypipes add an item to the infra meeting and then we call all be there at the same time and make a decision?15:06
yolandamm, shade with out namanger?15:06
yolandamanager?15:06
yolandado we want to go at that level?15:06
*** carl_baldwin_ has joined #openstack-infra15:06
mordredyolanda: I believe that we have to15:06
mordredyolanda: because otherwise once we move to shade we will lose the ability to have rate limiting context15:06
jogoanteaya: well hopefully jaypipes will just chime on on the patch itself15:07
anteayajogo: that works too15:07
mordredyolanda: since shade necessarily hides whether a given action is 0, 1, 2 or 10 REST API calls15:07
*** jlanoux_ has quit IRC15:07
mordredotherwise it's not very useful15:07
yolandamordred, i see... but actually i'm testing and that priority concept is proving to be useful15:07
jeblairjogo: if i thought these were completely different ideas, i would go with fungi's option -- new name.  if i thought that we were going to make a second attempt to do what we did the first time, i would keep it in repo and commit an rm -fr.  if i thought the previous effort was a mistake and so do the people working on the previous effort, i'd rename it out of the way and use the name on a new repo.15:08
jeblairjogo: hth15:08
yolandahow could we port that to shade if needed?15:08
jeblairjogo: what patch?15:08
mordredyolanda: well, we'll need to think about that15:08
*** zul has quit IRC15:08
jogojeblair: https://review.openstack.org/#/c/168883/15:08
*** zul has joined #openstack-infra15:08
*** jlanoux has joined #openstack-infra15:08
yolandagoing to stress hpcloud a bit more with that, i hope to don't break it :)15:09
mordredyolanda: good lukc!15:09
*** mrmartin has joined #openstack-infra15:09
fungijogo: great summary15:10
*** smccully has joined #openstack-infra15:11
*** ChuckC has joined #openstack-infra15:12
flip214fungi: thanks, but IIRC both referenced commits were relevant when I read his answer. Guess I'm wrong, though.15:12
fungiwow... jenkins thinks check-pbr-installation-dsvm is averaging 2 hours lately?15:12
*** harlowja_at_home has joined #openstack-infra15:13
clarkbflip214 it is relevant...15:13
*** Longgeek has quit IRC15:13
flip214clarkb: hi. it is? sorry, couldn't tell from the commit message.15:13
flip214all right, thanks.15:13
clarkbflip214 yes setting the guest cpu model is reauired for a working multinode cloud15:13
flip214I was afraid that the needed features got merged when I wasn't looking ;)15:13
*** radez_g0n3 is now known as radez15:13
clarkbhowever, please start with a single node test15:14
clarkbit will make everything simpler to start15:14
*** tonytan4ever has quit IRC15:15
*** afazekas has quit IRC15:16
*** tonytan4ever has joined #openstack-infra15:16
*** dannywilson has joined #openstack-infra15:16
claudiupopamordred: just to be sure, do you want this cherry pick to go through git review?15:18
*** baoli_ has quit IRC15:18
*** annegentle has quit IRC15:19
*** annegentle has joined #openstack-infra15:19
mordredclaudiupopa: yes. all changes must go throguh git review15:22
*** baoli_ has joined #openstack-infra15:22
*** dkranz has quit IRC15:22
claudiupopamordred: here it is https://review.openstack.org/#/c/168953/115:22
*** sputnik13 has joined #openstack-infra15:23
*** dulek has quit IRC15:24
*** shakamunyi has joined #openstack-infra15:25
*** chlong has quit IRC15:26
*** Ala has quit IRC15:26
*** sdake_ has joined #openstack-infra15:26
*** belmoreira has quit IRC15:27
*** hrubi has joined #openstack-infra15:29
*** tiswanso has quit IRC15:29
*** dboik_ has joined #openstack-infra15:30
*** baoli_ has quit IRC15:30
*** sdake has quit IRC15:30
*** baoli_ has joined #openstack-infra15:31
*** tiswanso_ has joined #openstack-infra15:31
*** MaxV has quit IRC15:32
*** e0ne is now known as e0ne_15:32
*** dboik has quit IRC15:32
*** ghostpl_ has quit IRC15:35
*** ghostpl_ has joined #openstack-infra15:36
*** dizquierdo has quit IRC15:37
*** e0ne_ is now known as e0ne15:38
*** dboik_ has quit IRC15:42
*** dboik has joined #openstack-infra15:43
*** otter768 has joined #openstack-infra15:43
*** deepakcs has quit IRC15:43
*** ddieterly has quit IRC15:43
clarkbmordred: I am not sure how 168634 addresses your concern over the weekend. Is it relying on the shade internal rate limiting that you linked earlier?15:46
*** zz_johnthetubagu is now known as johnthetubaguy15:46
clarkbmordred: the only problem with that is the rate limit won't know about any of the limiting that nodepool is doing internally but that is likely ok while the transition to shade happens15:46
openstackgerritCyril Roelandt proposed openstack-infra/project-config: saharaclient: enable Python3 jobs  https://review.openstack.org/16889915:47
openstackgerritMonty Taylor proposed openstack-infra/project-config: Add elements for Infra servers  https://review.openstack.org/14084015:47
mordredclarkb: you have to look at the two previous patches15:48
mordredclarkb: nodepool will pass in the nodepool providermanager as a TaskManager parameter15:48
*** otter768 has quit IRC15:48
mordredthat means that shade will use nodepool's taskmanager instead of its own task manager15:48
mordredclarkb: which means that any API tasks it runs will get done in the context of the nodepool ratelimiting15:48
mordredclarkb: but nodepool doesn't need to know the specifics about which shade API call run how many REST API calls15:49
clarkbI see15:49
zaromorning15:49
clarkbmordred: does that formalize the task manager in a way that may change (thinking about jeblair's change that made hpcloud have a sad)15:50
mordredclarkb: it would mean we need to define an interface for sure15:50
anteayamorning zaro15:50
mordredand there are additional design questions we'll want to think about - such as how would we handle things like what yolanda is trying to do15:50
*** jcoufal has quit IRC15:51
mordredbut basically, I would hope that at the very least it shows that we can have nodepool own the knowledge about how often it wants to do things, and still have shade own and hide the logic about which cloud things are needed for a particular action15:52
yolandaso at first tries i like what i see there, it's regularly triggering the polling tasks  and adding nodes, not waiting for the whole queue of create and deletes to finish15:52
clarkbmordred: yup the general idea seems good. I am just a bit concerend because we just tried to change this particular interface15:52
yolandamordred, you are right, shade should be dumb and nodepool should have the knowledge15:52
clarkbmordred: and trying to coordinate that with a library that is much more concerend about bcakward compatibility will only make it harder15:52
boris-42clarkb: hi15:52
*** shakamunyi has quit IRC15:52
clarkbboris-42: hello15:52
boris-42clarkb: could you please add me to nerdreviewer-cores https://review.openstack.org/#/admin/groups/654,members ?)15:53
openstackgerritMerged openstack-infra/elastic-recheck: Add query for horizon integration test bug 1436903  https://review.openstack.org/16875315:53
openstackbug 1436903 in OpenStack Dashboard (Horizon) "integration tests failing blocking gate" [Critical,Triaged] https://launchpad.net/bugs/1436903 - Assigned to David Lyle (david-lyle)15:53
clarkbboris-42: done15:54
boris-42clarkb: thank you)15:55
mordredclarkb: well, actually, I do not think the TaskManager interface is one that we would need to make one we'd care about - it shoudl be an impl detail that shade users do not think about except in extreme cases like nodepool15:55
mordredclarkb: in fact, we could make it a private function to add a taskmanager and put big warning signs "this is onlly to be used by nodepool, don't use it if you don't know what you're doing"15:56
*** jcoufal has joined #openstack-infra15:56
*** amitgandhinz has joined #openstack-infra15:56
jogohave all the nodepool zombie instances been cleaned out and we are actually using both clouds now?15:56
mordredclarkb: (just thinking out loud)15:56
*** jcoufal has quit IRC15:56
mordredclarkb: the important parts of the API for shade and backwards compat woudl be the parts we expect people to program to - which is definitely not the internal task interface15:56
silehtI have a review where I have used 'Depends-On' the dependencies have been merged, but this one is never processed by zuul: https://review.openstack.org/#/c/166763/15:57
clarkbmordred: I am just thinking in the nodepool case. eg if we have to backpedal on a task manager change like that again it will require either a downgrade of shade too or a roll forward of shade15:57
*** ddieterly has joined #openstack-infra15:57
sileht'recheck' can help in this case ?15:57
*** scheuran has quit IRC15:57
*** markus_z has quit IRC15:57
mordredclarkb: totally. it's definitely a think we'll need a plan on15:58
clarkbsileht: if that change was approved before its deps then zuul would not have processed it once the deps did merge15:59
clarkbsileht: a recheck should get the ball rolling15:59
clarkbsileht: actually hrm15:59
silehtclarkb, thanks I try15:59
clarkbbecause the verified vote is already a +1 that may not work15:59
anteayaclarkb: yeah I was looking at that too15:59
clarkbzaro: did you ever get traction on your gerrit change to fix ^15:59
clarkbsileht: instead a reapproval may be best15:59
anteayaclarkb: I'm inclined to say a new +A15:59
clarkbanteaya: yup15:59
marcusvrn1asselin kaisers1: hi! I was out of office, then I couldn't attend to the meeting...but have read the log and I have also a problem with test_minimum_basic_scenario16:00
*** tonytan4ever has quit IRC16:00
*** dhritishikhar_ has joined #openstack-infra16:00
silehtclarkb, I will ping a core dev, so16:00
anteayamarcusvrn1: can you post a stacktrace?16:00
*** gokrokve_ has joined #openstack-infra16:00
* asselin looks up his log of that too16:00
kaisers1marcusvrn1: ah, ok16:00
*** Guest30324 has quit IRC16:00
marcusvrn1anteaya: sure, one sec...16:01
anteayathanks16:01
nibalizerfbo: hi16:01
kaisers1marcusvrn1: asselin: Are these similar issues or different issues with the same test?16:01
mordredclarkb: although - actually, the only thing in the change we just made and reverted that would have been shade-side would have been in Task - and even that was an impl detail change - the real guts f the change was TaskManager's logic - so that really could have been done almost totally with a change to nodepool16:01
*** YorikSar has quit IRC16:01
nibalizerif you have some ideas on how to improve the apply-test im all ears16:01
mordredeven in the new patches16:01
*** Hal has joined #openstack-infra16:02
*** tqtran has joined #openstack-infra16:02
anteayakaisers1: hence the request for some stacktraces16:02
clarkbmordred: I think the task interface changed too16:02
zaroclarkb: no, only got one review.  maybe you can post a review?.16:02
mordredclarkb: all we need to do is _add_ a Task api that lets a task manager indicate it wants the other behavior - and nodepool's TaskManager could use either16:02
*** Hal is now known as Guest1782916:02
anteayazaro: link?16:02
clarkbzaro: sure, have a link handy?16:02
mordredclarkb: it did - but the Task interface change can be done in a way that can do both things16:02
mordredclarkb: (was just looking at the code)16:02
*** gokrokv__ has joined #openstack-infra16:02
*** gokrokve_ has quit IRC16:02
zaroclarkb: https://gerrit-review.googlesource.com/6553016:03
marcusvrn1kaisers1: The error that I got is related to the SSH connection timeout16:03
Shrewsmordred: you were busy this weekend16:03
marcusvrn1anteaya: kaisers1: http://paste.openstack.org/show/197584/16:03
asselinmarcusvrn1, anteaya kaisers1 I have same issue as marcusvrn1 http://paste.openstack.org/show/197586/16:04
*** gokrokve has quit IRC16:04
anteayaawesome16:04
anteayayou may have found a bug16:04
anteayawell done16:04
marcusvrn1asselin: the same error here16:05
kaisers1Mine seems to be different, I'll dig deeper... :)16:05
kaisers1probably i'll hit the same once i'm done with my issue...16:05
marcusvrn1kaisers1: yes16:05
clarkbzaro: I really dislike the gerrit overriding ^F in my browser behavior...16:05
mordredclarkb: anyway -I'm nt saying that change would have worked, or that we don't need to do some deep thinking - but looking at the change itself I can see how we could make it so that the forward/back changes were contained within nodepool still16:05
anteayaso now the question is, is it a bug with the test or has the test actually turned up a bug in cinder code16:05
anteayamarcusvrn1 asselin kaisers1 do one of you want to start an etherpad to keep track of links so that you can work together on this?16:06
*** annegentle has quit IRC16:07
asselinanteaya, the test runs as part of cinder in the gate16:07
*** unicell has quit IRC16:07
anteayaasselin: and it is passing in the gate?16:07
anteayaso then why is it failing for two ci systems?16:07
marcusvrn1anteaya: asselin: another test returns the same error, but it's intermittent...16:08
marcusvrn1this one: tempest.thirdparty.boto.test_ec2_instance_run.InstanceRunTest.test_compute_with_volumes16:08
fungijogo: it looks like once our utilization of hpcloud picked back up for the day, we've got 180+ nodes which have been in a "building" state for more than half an hour and all are in hpcloud16:08
anteayamarcusvrn1: which again could be a bug, but a race condition16:08
*** ssam2 has quit IRC16:09
jogofungi: too bad thanks.16:09
asselinanteaya,  drivers exposes other issues such as timing. Default driver (gate) is lvm with is run locally on the same instance.16:09
*** shakamunyi has joined #openstack-infra16:10
*** thedodd has quit IRC16:10
anteayaasselin: ah16:10
anteayaasselin: and yes timing issues are worth looking at16:10
*** yfried has joined #openstack-infra16:13
marcusvrn1asselin: hmmm may be that's why gate doesn't fail16:13
*** ssam2 has joined #openstack-infra16:13
* asselin will be back in 45 mins16:13
fungijogo: though it looks like in the last few minutes hpcloud has started returning more ready nodes, so maybe it's better behaved than last week16:14
clarkbzaro: commented, one inline thing but not familiar enough with the code to know if its a real -116:14
fbonibalizer, hi yes to be short I think we can use lxc on virtual node to spawn some precise,trusty,centos containers and run the manifests based on a custom manifests/site.pp and hiera db filled with fake config/key pairs in order to have an almost complete CI up and running16:14
fbonibalizer, then we can add some smoke testing with tools like serverspec to check services are correctly setup16:15
clarkbmordred: shade still needs to know what the Task interface is though right? eg it has to pick one of the options that may be available to it? in any case I am sure this is doable, just making sure we think about the backward compat requirements around it16:16
nibalizerfbo: yea i've done some stuff like that16:16
clarkbmordred: I don't think its too far fetched to say if nodepool requires this then any full features shade consumer will likely need it too16:17
nibalizer(outside of openstack testing i should say)16:17
jogofungi: good to hear16:17
nibalizerthere are some things that we manage that don't or sometimes dont work in lxc containers/docker containers like iptables16:17
nibalizerbut we can get there when we get there16:17
nibalizeri was thinking to use envassert instead of serverspec since it is python16:17
*** asselin has quit IRC16:19
fbonibalizer, same for me I have some experience with LXC to test deployment based on puppet for tool like gerrit/jenkins/zuul but not with the system-config manifest16:19
*** armax has quit IRC16:19
yolandaso mordred, the stress test was quite good, trying to generate some charts about that16:19
fboYes if we can skip ruby stuff I'll be happy, I'll look at envassert then :)16:20
mordredclarkb: I think it's less likely. ansible, for instance, wants to have nothing to do with knowing about this. (and I had to make zero changes to ansible for it to continue working) BUT - I'm on the phone, so I'll sync back up on it in about an hour16:20
fbonibalizer, so what do you think if we merge efforts to bring something in openstack-infra to perform more in depth testing of the manifests ?16:21
*** ssam2 has quit IRC16:24
clarkbyolanda: for the priority queue work shouldn't most of those hit the internal cache instead?16:26
yolandawell, the request is only generated when the cache is missing16:26
clarkbyolanda: right, so 99% of the time the cache will be hit and we keep moving?16:26
yolandaso it just generates that api call and puts that as high priority, so the polling is run on time16:26
yolandawell, it's not what happens really... the cache interval is quite short16:26
yolandaand under load, the create, delete nodes tasks are just queued and preventing the polling tasks to run16:27
yolandai've seen cases of 15-20 minutes without this task being executed16:27
nibalizerfbo: i dont really have an efforts in flight16:27
clarkbyolanda: the polling tasts are?16:27
nibalizerthere is one patch but it doesn't go close to what you're talking about16:27
yolandagetserverlists, getfloatingiplists...16:27
*** ajmiller has joined #openstack-infra16:28
*** dkranz has joined #openstack-infra16:28
yolandaclarkb, so from our side, the problem was that nodes were created, but as the getserverlists wasn't called, system wasn't able to asign a floating ip . Even when that floating ip was assigned, the floatingiplist task was taking 10 minutes to run, so the node was idle there16:28
yolandaclarkb, it comes a bit with a conversation we had with mordred and jeblair on friday, talking about having 2 different managers: one for lists tasks, one for the create/delete... but i think that this priority makes thing simpler and it works16:29
*** baoli_ has quit IRC16:29
clarkbyolanda: basically the cache update period is > than the cache timeout period16:29
fbo nibalizer ok I'm writing a spec to describe that so maybe it can be a good starting point to go deeper in order to discuss and defined something ?16:29
yolandamuch greater, yes16:30
*** prmtl_ has quit IRC16:30
yolandaclarkb, but it is nothing we can control the way nodepool it's now, because it depends on the rate and the number of api calls queued16:30
*** e0ne has quit IRC16:30
yolandaso that's my reason for the priority, so we make sure that this interval is accomplished16:30
clarkbyolanda: right and thats a FIFO with expensive tasks inline16:30
*** ssam2 has joined #openstack-infra16:30
yolandaclarkb, that should be implemented as a heapq in python?16:31
nibalizerfbo: good idea16:31
nibalizerfbo: also I think the openstackci module is a good place to jump off our testing16:32
clarkbyolanda: maybe? my only concern here is that the list servers/fips/images/etc may starve the creates and deletes16:32
nibalizersince that will be a known excpected state with a lot of complexity16:32
yolandaclarkb, no, because they rely on cache16:32
yolandaso you won't be duplicated calls, only once at a time that cache expires16:32
*** pblaho has quit IRC16:32
*** hashar has quit IRC16:32
clarkbyolanda: and without updating the priorities of any tasks based on age (or some other criteria) we have no way of ensuring that the other tasks are serviced16:32
*** aysyd has quit IRC16:32
clarkbyolanda: thats only true if the api responds within a cache timeout period right?16:33
clarkbyolanda: and I think we are already saying that this may not be the case? though I guess that is not the case or other api calls16:33
clarkbsince the listings don't starve themselves, the deletes starve them16:33
jeblairyolanda, clarkb: what priority thing?16:33
*** marun has joined #openstack-infra16:33
clarkbjeblair: 16868616:33
yolandajeblair https://review.openstack.org/#/c/168686/16:33
*** esker has joined #openstack-infra16:33
*** esker has quit IRC16:34
yolandaclarkb, let me check about the caching implementation16:34
openstackgerritSpencer Krum proposed openstack-infra/system-config: Print the node definition before apply test  https://review.openstack.org/16734616:34
yolandaalthough this will be moved to shade so it will be different16:34
*** samueldmq has quit IRC16:34
*** emagana has joined #openstack-infra16:34
*** amitgandhinz has quit IRC16:35
*** amitgandhinz has joined #openstack-infra16:35
yolandaclarkb, so server lists and floating ip lists have an interval of 5 secons16:35
jeblairyolanda, clarkb: the default cache interval is 5 seconds -- hpcloud was taking 20-30 seconds for a create 2 weeks ago, and 40-60 seconds for a create last week16:35
yolandaseconds16:35
*** marcusvrn1 has quit IRC16:35
*** marcusvrn1 has joined #openstack-infra16:35
yolandajeblair, so as per our friday conversation, i thought into a different approach, that was that priorityqueue thing16:35
*** aysyd has joined #openstack-infra16:35
fbonibalizer, yes you are right ! I'll mention in the spec.16:36
jeblairyolanda, clarkb: i believe that means that create and deletes will be starved until all existing create/deletes are complete16:36
yolandai've been testing that on my hpcloud account and results were better16:36
nibalizerfbo: 167346 is my current set of changes to the apply test, just trying to split it up and make it easier to read16:36
mordredI believe the create/delete speed is going to be dependent on the tenant16:36
jeblairmordred: yes, and change16:36
mordredbecause of soft deletes and quota calculation16:36
yolandaas it's getting the created servers and floating ips as they come16:36
jeblairyolanda: how do you define better?  i believe it _will_ make a single bringup faster.  but it will lower overall throughput.16:36
fbonibalizer, thanks I'll have a look16:36
jeblairyolanda: i think we care more about overall throughput16:36
yolandajeblair, i'm working on some carts16:36
yolandacharts16:36
fungiit's sort of like measuring resistance across a circuit with a trickle current rather than under load. you can get entirely different readings16:37
jeblairya16:37
yolandajeblair, well, my thought is: you have ready nodes without a floating ip, because the serverlists task hasn't come. That means idle nodes , increasing quota and doing nothing16:37
fungii've seen hpcloud perform just fine over the weekend when we ask very little of it, then crumble again when the week starts16:37
jeblairfor our workload, i think we want to know how many servers we can create and delete over a multi-hour period16:37
yolandaif you are able to get that nodes working as they come, you will gain time16:38
*** harlowja_at_home has quit IRC16:38
*** baoli has joined #openstack-infra16:38
yolandawe've had cases of serverlists not executing on 15 minutes, you said that as well on friday. So it means 15 minutes for a node waiting there without activity and wasting quota16:38
jeblairyolanda: yes that is true when you look at that single node.  but when you look at hundreds of them, you see that you can only create and delete them at a fixed, slow, rate.  so the overall throughput of the system is defined by that rate.16:38
*** unicell has joined #openstack-infra16:39
clarkbyolanda: I think thats more of an issue around atomicity or requests and less about priority16:39
clarkbbasically we want atomic work units that result in "working node" "node removed"16:39
clarkbinstead of 5 api calls for each thing all dependent on each other16:39
jeblairyolanda: yeah, which is why i suggested separating the providers so that you maximize the throughput of the slow calls and the fast ones.16:40
*** jlvillal_ has joined #openstack-infra16:40
jeblairyolanda: however, if we go with the priority queue as written, we will actually just reverse the situation16:40
yolandajeblair, but the lists task come on a controlled rate due to caching16:41
jeblairyolanda: also, splitting the providers means separating calls that require the quota lock and ones that don't.16:41
*** sergsh has quit IRC16:41
yolandajeblair, do we have calls that don't need a rate?16:41
*** jlvillal_ has left #openstack-infra16:41
jeblairyolanda: not to my knowledge (though implementation my vary by provider)16:42
*** ghostpl_ has quit IRC16:42
jeblairyolanda: hpcloud is behaving pathologically bad16:42
jeblairi am not certain we should try to engineer a complete solution for this behavior16:42
greghaynesclarkb: Is what youre getting at that essentially that we dont mind starving list if its not going to actually result in getting a resource sooner?16:43
*** MaxV has joined #openstack-infra16:43
yolandaok, let me work on the provider approach as well, it's not complicated16:43
yolandabut wanted to show and test that as well16:43
yolandawe have some tooling to analyze it so we could test both16:43
jeblairyolanda: okay.  i'd love to see data from this approach too16:43
*** Somay has joined #openstack-infra16:43
clarkbgreghaynes: yes16:43
jeblairyolanda: yeah, i think there's stuff we can learn from here, but i have concerns about it :)16:44
clarkbgreghaynes: because without atomic work unit api calls you have a bunch of calls scattered through that queue and the list* calls are only a portion of that16:44
greghaynesok, I think thats key - and preserving insertion ordering should inherently do that16:44
jeblairyolanda: but i think we will need to know how it performs at scale -- with at least 100 nodes over several hours16:44
*** crc32 has joined #openstack-infra16:44
yolandayes, i just wanted to leave that running overnight16:44
openstackgerritMerged openstack-infra/nodepool: Correct diskimage-builder name usage  https://review.openstack.org/16777116:44
jeblairyolanda: if you don't have the quota for that, you should be able to create a simulation by using the timing data in the nodepool log16:44
*** yamahata has joined #openstack-infra16:44
*** jistr has quit IRC16:44
clarkbgreghaynes: instead we need to either process work units together internal to nodepool or fix nova/neutron to give us apis that represent logical changes16:45
yolandajeblair, i got a magical quota increase this afternoon :)16:45
jeblairyolanda: since you know that under load it takes X seconds for a create call, and Y seconds for a list ip call16:45
jeblairyolanda: can you use 100 nodes without affecting infra or gozer?  i thought hpcloud was under a lot of capacity pressure?16:45
*** isviridov has joined #openstack-infra16:45
yolandajeblair, well, at least i've been doing that for half an hour16:45
yolandaand it was doing the job16:46
nibalizeryolanda: in us east or a different region?16:46
greghaynesclarkb: Yep, thats the basis of a lot of modern scheduling problems ;)16:46
yolandaus east16:46
jeblairclarkb: i would love atomic work units from openstack, but by not making them atomic in nodepool we are able to be much more efficient16:46
greghaynesclarkb: inserting into work queue in order is a quick way to get that though16:46
*** ghostpl_ has joined #openstack-infra16:46
yolandai have over 100 instances creted now16:46
clarkbgreghaynes: it is a queue so it already is a fifo16:46
clarkbgreghaynes: the problem is you have more than one input16:47
greghaynesoh, its mutiprocess16:47
yolandaso i'll let it run until night, and will grab the logs, and tomorrow i can test with the independent provider approach16:47
*** MaxV has quit IRC16:47
yolandadon't want to let them run on the night so i crash something and i'm not awake...16:47
clarkbjeblair: ya, I don't think there is a major problem with what we do now. Its easier for me to think about this problem in the context of scheduling logical work units than it is "prioritize list servers"16:47
jeblairyolanda: okay.  while you're waiting, you could try writing a program to simulate it much faster :)16:48
clarkbgreghaynes: its multi threaded16:48
greghaynesactually, I think what you have still gets what youre describing - something is necessarially blocking on the task in the queue16:48
greghaynesit just might not be the most important task, which is a different problem16:48
jeblairyolanda, clarkb: so i think the starvation potential with the priority queue is more nuanced than what i described before...16:48
*** jaypipes has quit IRC16:49
yolandawhat do you mean?16:50
*** MaxV has joined #openstack-infra16:50
jeblairyolanda, clarkb: i think what would be required for it to happen is for a list servers and list floating ips task to get into the queue around the same time.  if the second one takes more than 5 seconds, then the next list <something> task will get promoted.  and that's how we starve create and delete.16:50
yolandayou are right with that16:51
clarkbjeblair: yes that is what I was trying to describe earlier without specific cache timeout numbers16:51
jeblairyolanda, clarkb: so the requirement for starvation is that ether list servers or list floating ips takes more than the cache interval.16:51
*** Swami has joined #openstack-infra16:51
jeblairclarkb: sorry, i'm catching up :)16:51
*** ildikov has joined #openstack-infra16:52
jeblairyolanda: so anyway, let's collect data and think about this, but even if it looks better, we need to carefully consider it because if listing servers currently takes 4.9 seconds, it could mean that performance drastically changes next week when it takes 5.1 seconds.16:52
yolandajeblair, and we could prevent in some way to launch a request if one was already launched before?16:53
jeblairyolanda: well, that's what the cache does.  we could increase the timeout, but that will have ripple effects (making the whole system slower again)16:54
openstackgerritMichael Krotscheck proposed openstack-infra/storyboard: Fixed import  https://review.openstack.org/16846716:54
yolandathat's a complicated balance :)16:54
jeblairya16:54
yolandait we had callbacks of some way we could just stop the polling ones...16:54
mordredwe don't16:54
mordredsadly16:55
mordredbut yes16:55
mordredit would be so great if openstack had a user-facing callback mechanism16:55
yolandamodred, ask for it :)16:55
*** dizquierdo has joined #openstack-infra16:55
mordredyolanda: heh. I'm just focusing on getting ssh host keys out of nova for next cycle - I think a generalized callback mechanism might take a while :)16:56
*** dizquierdo_ has joined #openstack-infra16:57
*** derekh has quit IRC16:57
*** wicope has joined #openstack-infra16:58
*** AJaeger_ has joined #openstack-infra16:58
*** AJaeger has quit IRC16:58
*** erlon has joined #openstack-infra16:59
*** YorikSar has joined #openstack-infra16:59
*** esker has joined #openstack-infra16:59
*** dizquierdo has quit IRC17:00
*** dkranz has quit IRC17:01
*** ssam2 has quit IRC17:01
*** btran has joined #openstack-infra17:02
*** AJaeger has joined #openstack-infra17:03
*** esker has quit IRC17:03
*** AJaeger has quit IRC17:03
*** AJaeger has joined #openstack-infra17:03
*** Guest17829 has quit IRC17:03
*** sarob has joined #openstack-infra17:03
*** adalbas has joined #openstack-infra17:04
*** mpavone has quit IRC17:04
*** Hal has joined #openstack-infra17:04
*** bknudson has quit IRC17:04
*** Hal is now known as Guest9464717:05
*** tiswanso_ has quit IRC17:05
*** sarob_ has joined #openstack-infra17:06
*** davideagnello has joined #openstack-infra17:06
*** hdd has joined #openstack-infra17:06
AJaeger_jogo, jeblair: REgarding gantt: Is then https://review.openstack.org/#/c/168682/ the right approach? Or should I abandon both for now and we leave status quo?17:08
*** MaxV has quit IRC17:08
*** sarob has quit IRC17:10
*** Somay has quit IRC17:11
openstackgerritMichael Krotscheck proposed openstack-infra/storyboard: Remove page_size_maximum  https://review.openstack.org/16763317:11
openstackgerritMichael Krotscheck proposed openstack-infra/storyboard: Remove page_size_default  https://review.openstack.org/16765217:11
*** HeOS has quit IRC17:11
openstackgerritSpencer Krum proposed openstack-infra/system-config: Add a helpful friend to gerrit reviews  https://review.openstack.org/16901017:13
*** pal_ has joined #openstack-infra17:13
openstackgerritMichael Krotscheck proposed openstack-infra/storyboard: Fixed import  https://review.openstack.org/16846717:13
*** ssam2 has joined #openstack-infra17:14
openstackgerritMichael Krotscheck proposed openstack-infra/storyboard: Remove page_size_maximum  https://review.openstack.org/16763317:14
openstackgerritMichael Krotscheck proposed openstack-infra/storyboard: Remove page_size_default  https://review.openstack.org/16765217:14
*** armax has joined #openstack-infra17:16
*** tsg_ has joined #openstack-infra17:16
*** dkranz has joined #openstack-infra17:16
*** ssam2 has quit IRC17:17
*** sarob_ is now known as sarob17:17
*** yamahata has quit IRC17:18
*** dtantsur is now known as dtantsur|afk17:18
*** Somay has joined #openstack-infra17:20
*** MarkAtwood has quit IRC17:20
*** peristeri has quit IRC17:21
*** Somay has quit IRC17:22
*** prmtl has joined #openstack-infra17:22
*** jgrimm is now known as zz_jgrimm17:22
*** sarob_ has joined #openstack-infra17:24
*** achanda has joined #openstack-infra17:24
*** sarob has quit IRC17:24
*** melwitt has joined #openstack-infra17:25
*** harlowja_away is now known as harlowja17:25
*** tnovacik has quit IRC17:25
*** gokrokve has joined #openstack-infra17:25
*** sarob has joined #openstack-infra17:25
*** amitgandhinz has quit IRC17:25
*** samueldmq has joined #openstack-infra17:25
zaroclarkb: replied to your question.  i agree with you about the ^f i'm guessing that it's because it works well on macs because there's the command-f key.17:26
*** tiswanso has joined #openstack-infra17:26
arosenquick question. to push a git tag the wiki says: git push gerrit tag version; what is the remote of gerrit?17:27
clarkbzaro: fwiw I don't think they should be doing different things, they should do the same exact thing so that the behavior is always the same for stream events17:27
clarkbarosen: if you use git-review it is 'gerrit'17:27
SpamapSmordred: regarding user facing callback mechanisms: Zaqar was supposed to be the facilitator for that. But it got pounded on. :-P17:27
arosenclarkb:  i do use git review but I guess that dosen't set my .git/config to gerrit ?17:27
*** MarkAtwood has joined #openstack-infra17:27
clarkbarosen: do a `git remote -v`17:28
*** dkranz has quit IRC17:28
*** e0ne has joined #openstack-infra17:28
clarkbSpamapS: I thought zaqar specifically didn't want to be in cloud17:28
arosenclarkb:  doh i see.17:28
arosenCreating a git remote called "gerrit" that maps to:17:28
arosenssh://arosen@review.openstack.org:29418/stackforge/python-congressclient.git17:28
*** gokrokv__ has quit IRC17:28
arosenthanks17:29
clarkbSpamapS: that was a huge point of confusion and discussion17:29
*** sarob_ has quit IRC17:29
*** ivar-lazzaro has joined #openstack-infra17:29
*** gokrokve has quit IRC17:30
*** ivar-lazzaro has quit IRC17:30
clarkbjogo: I think the tests that failed on my 64bit guest cpu change may have done so because that cpu isn't "real" enough to actually log into the VM17:30
prmtlfolks, I kindly ask for a review (and hopefully merge): https://review.openstack.org/#/c/167539/17:30
*** ivar-lazzaro has joined #openstack-infra17:30
clarkbjogo: but knowing what I know now we can do a core2duo - monitor - pse36 and that *should* work so going to try cleaning that up this morning17:31
*** julim has quit IRC17:31
*** hodos has joined #openstack-infra17:31
SpamapSclarkb: it was supposed to be available to in and out17:31
SpamapSclarkb: so that was supposed to be where things like Heat and Nova would put messages about events users had requested.17:32
SpamapSI believe Heat still wants to do that.17:33
SpamapSand may even drive the effort into the others.17:33
*** julim has joined #openstack-infra17:33
*** jerryz_ has quit IRC17:33
openstackgerritMichael Krotscheck proposed openstack-infra/storyboard-webclient: Removed grunt:server  https://review.openstack.org/16316517:34
*** mriedem has quit IRC17:34
fungiarosen: yep, see the first note at http://docs.openstack.org/infra/manual/drivers.html#tagging-a-release17:34
zaroclarkb: probably agree with you there, but didn't want to change the existing flow too much for that change.17:34
*** ghostpl_ has quit IRC17:34
clarkbzaro: ok17:35
*** dboik has quit IRC17:35
*** e0ne is now known as e0ne_17:35
*** dboik has joined #openstack-infra17:35
*** crc32 has quit IRC17:36
*** tonytan4ever has joined #openstack-infra17:36
zaroclarkb: too bad the shortcut keys are not confiugrable. that would be a nice feature17:36
*** amotoki has quit IRC17:36
*** annegentle has joined #openstack-infra17:37
*** Guest94647 has quit IRC17:38
yolandaSpamapS, that looks as an interesting feature, didn't know about that effort17:38
*** yamahata has joined #openstack-infra17:39
*** amitgandhinz has joined #openstack-infra17:39
*** e0ne_ is now known as e0ne17:39
zaroclarkb: looks like someone created a patch for that but, it's on the android fork of gerrit but I can't access it. Argg! probably in draft mode.17:39
*** aduarte has joined #openstack-infra17:41
*** ZZelle_ has joined #openstack-infra17:41
*** dkranz has joined #openstack-infra17:41
*** jaypipes has joined #openstack-infra17:42
*** annegentle has quit IRC17:43
*** otter768 has joined #openstack-infra17:44
*** annegentle has joined #openstack-infra17:44
*** MaxV has joined #openstack-infra17:44
openstackgerritMerged openstack-infra/project-config: Add CI jobs for fuel-tasklib  https://review.openstack.org/16753917:46
*** wicope has joined #openstack-infra17:46
*** ihrachyshka has quit IRC17:47
*** otter768 has quit IRC17:48
*** Sukhdev has joined #openstack-infra17:48
*** e0ne is now known as e0ne_17:49
SpamapSyolanda: I think it would be massively useful if a user could subscribe to status changes for an individual instance.17:51
*** mriedem has joined #openstack-infra17:51
SpamapSyolanda: and just open up a longpoll http request to receive notifications.17:51
*** prmtl has quit IRC17:51
SpamapSor a gearman connection. :)17:51
*** peristeri has joined #openstack-infra17:52
openstackgerritAndrey Pavlov proposed openstack-infra/project-config: Improve ec2-api gating jobs for rally testing  https://review.openstack.org/16664817:53
*** e0ne_ has quit IRC17:54
*** pal_ has quit IRC17:55
*** dustins_ has joined #openstack-infra17:56
*** dustins_ has quit IRC17:57
jogoAJaeger_: not sure myself actually17:57
jogoJay's  trademark concern over gantt make me thing the attic is a good solution17:58
*** sarob has quit IRC17:58
AJaeger_jogo: we don't need to force a solution yet, I was just trying to clean this up,...17:59
*** dustins has quit IRC17:59
jogoAJaeger_: yeah thanks for doing this. It will be good to figure out an answer18:00
*** dustins has joined #openstack-infra18:01
clarkbjogo: trademark concern? A quick uspto search doesn't really bring anything up18:02
jogoclarkb: I was taking jaypipes comment on face value, not sure what he was referring to18:02
fungimore like contention over the term "gantt" (because gantt charts are a thing)18:02
clarkbjogo: oh is there a comment? /me digs through sb for change link18:03
*** hodos|2 has joined #openstack-infra18:03
*** ghostpl_ has joined #openstack-infra18:03
clarkbneat uspto shows you liveness and abandonment dates18:04
clarkbI wonder how up to date that info is18:04
jaypipesclarkb: I commented that I would be surprised if the project name "gantt" didn't get messy, legally18:04
*** teran_ has quit IRC18:04
*** teran has joined #openstack-infra18:04
*** hodos has quit IRC18:05
*** tsg_ has quit IRC18:05
*** tsg has joined #openstack-infra18:05
*** greghaynes has quit IRC18:07
yolandaSpamapS, yes, i'd love to have some way of callbacks, but this doesn't sound like an easy task18:09
*** jlanoux has quit IRC18:09
*** teran has quit IRC18:09
*** bknudson has joined #openstack-infra18:10
*** dhritishikhar_ has quit IRC18:11
*** thingee has joined #openstack-infra18:11
*** bknudson has left #openstack-infra18:12
*** greghaynes has joined #openstack-infra18:13
*** dizquierdo_ has quit IRC18:14
*** bknudson has joined #openstack-infra18:14
*** svarnau has joined #openstack-infra18:18
svarnauIs bug-tracking for zuul still on storyboard?  I thought I filed a bug, but I see no trace of it.18:22
*** esker has joined #openstack-infra18:22
openstackgerritMichael Krotscheck proposed openstack-infra/storyboard-webclient: Typo in Project Groups.  https://review.openstack.org/16903718:23
openstackgerritMichael Krotscheck proposed openstack-infra/storyboard-webclient: Project Group detail page improvements.  https://review.openstack.org/16903818:23
*** johnthetubaguy is now known as zz_johnthetubagu18:23
*** dizquierdo has joined #openstack-infra18:23
fungisvarnau: default storyboard page size is small and the ui currently doesn't give you any indication it's exceeded (pagination support hasn't been completed yet). check your storyboard account prefs and increase your minimum page size18:25
svarnaufungi: Thanks! that was indeed the problem18:26
*** gokrokve has joined #openstack-infra18:26
*** crc32 has joined #openstack-infra18:26
*** _nadya_ has quit IRC18:26
*** ddieterly has quit IRC18:27
*** Swami has quit IRC18:28
*** Swami has joined #openstack-infra18:28
*** svarnau has quit IRC18:30
*** MaxV has quit IRC18:31
*** e0ne has joined #openstack-infra18:34
*** ddieterly has joined #openstack-infra18:36
*** achanda has quit IRC18:38
*** HeOS has joined #openstack-infra18:39
openstackgerritMatthew Treinish proposed openstack-infra/project-config: Use localrc passthrough for tempest no admin job  https://review.openstack.org/16904218:39
mtreinishsdague: ^^^ will that do the right thing?18:39
*** pal_ has joined #openstack-infra18:40
sdaguemtreinish: nope18:40
openstackgerritJoe Gordon proposed openstack-infra/project-config: Move aiopcpu to nova check queue  https://review.openstack.org/16904318:40
mtreinishheh, figured18:40
jogoarmax sdague: ^18:41
jogodansmith: ^18:41
sdaguemtreinish: DEVSTACK_LOCAL_CONFIG18:41
mtreinishah, ok thanks18:41
*** carl_baldwin_ has quit IRC18:41
sdaguejogo: is that non-voting?18:41
*** dustins has quit IRC18:42
armaxjogo: thanks for the heads-up18:42
sdagueanyway, got to run for a bit18:42
jogosdague: yes, but let me double check18:43
*** dhritishikhar_ has joined #openstack-infra18:43
jogosdague: yes it is non voting (line 694)18:44
openstackgerritMatthew Treinish proposed openstack-infra/project-config: Add os-testr repo  https://review.openstack.org/16724318:44
openstackgerritMatthew Treinish proposed openstack-infra/project-config: Use localrc passthrough for tempest no admin job  https://review.openstack.org/16904218:45
*** ivar-lazzaro has quit IRC18:46
*** spzala has quit IRC18:46
*** xyang1 has quit IRC18:46
*** dustins has joined #openstack-infra18:48
*** pal_ has quit IRC18:51
*** Hal has joined #openstack-infra18:52
*** Hal is now known as Guest6506118:53
*** Rockyg has joined #openstack-infra18:57
*** achanda has joined #openstack-infra18:57
*** achanda has quit IRC18:57
*** achanda has joined #openstack-infra18:58
*** dizquierdo has quit IRC19:00
*** baoli has quit IRC19:00
*** xyang1 has joined #openstack-infra19:01
*** tkelsey has quit IRC19:01
openstackgerritMatthew Treinish proposed openstack-infra/project-config: Add os-testr repo  https://review.openstack.org/16724319:02
clarkbmordred: for 168601 why wouldn't nodepool provide that info to shade using its existing config?19:04
clarkbmordred: hopefully the use of shade should be transparent to nodepool users19:05
*** tnovacik has joined #openstack-infra19:06
*** claudiub has joined #openstack-infra19:07
openstackgerritArmando Migliaccio proposed openstack-infra/project-config: Enable DVR for Neutron aiopcpu job  https://review.openstack.org/15841519:08
armaxclarkb, jogo: ^^^19:08
*** spzala has joined #openstack-infra19:08
armaxclarkb: that was fast!19:08
mordredclarkb: it can continue to do that - nothing will break19:09
clarkbmordred: then can we abandon that change stack?19:10
mordredclarkb: however, there are additional things that we can get by starting to use the clouds.yaml file19:10
mordredno19:10
mordredwe want it19:10
mordredbut it's not going to break other people19:10
clarkbwhy do we want it? can you make sure that is explained in the commit messages (I am pretty sure it is not)19:10
clarkbmordred: I was hoping to avoid this philisophica ldiscussion but I think that users should never need to configure libraries directly when using libs in applications19:11
clarkbmordred: this is a major issue I have with how oslo.config works19:11
mordredsure. I'll go rework the config file - tl;dr - if we have the file, then when the openstackclient patch lands we can use with openstackclient or ansible on that host to perform any ad-hoc maint tasks without having to duplicate auth config19:11
mordredclarkb: what?19:11
clarkbmordred: as a nodepool user that wants features you shouldn't need to write multiple config files19:12
mordredI agree19:12
mordredand you don't  have to19:12
clarkband more importantl you should need to know that shade ever exists and has special config options19:12
clarkb*shouldn't need to know19:12
mordredyou don't have to19:12
mordredHOWEVER - if you want to _Also_ take advantage of teh fact that there is a config file that configures several other openstack utilities as well, you can opt in to doing so19:12
clarkbmordred: so this is purely for convenience and has nothing to do with nodepool?19:13
mordredwhich is why we want it19:13
mordredyes19:13
mordrednodepool does not need to require a clouds.yaml19:13
clarkbmordred: ok that isn't clear at all given that 168602 exists19:13
mordredif we, or anyone else, already has named configured clouds on their system beacuse they want to use those features, they can use them19:13
mordredclarkb: fair. I'll rework the commit message19:14
clarkbmordred: and yes oslo.confg has us generating environment specific config files because each time *client changes your config changes too19:14
clarkbmordred: its a bit crazy19:15
mordredclarkb: I totally agree with that19:15
*** hashar has joined #openstack-infra19:15
mordredclarkb: this is more "there are 5 pieces of information about your clouds that multiple tools need to know to operate, so if we put those informations in a sane place and have all the tools be able to opt-in to referring to a named cloud, it reduces the duplicative places we need to express those informations"19:15
mordredbut I think opt-in is key19:15
clarkbyes optin is good19:16
mordredthe openstackclient patch, for instance, allows you do say "openstack --os-cloud=foo servers list"19:16
mordredand will read the clouds.yaml file to figure out which things you want19:16
mordredbut also still just works with OS_ env vars or command line options19:16
mordredif doing that isnt' somethign you're interested in19:16
clarkbgotcha19:17
stevemarmordred, i'm hoping to take a deeper look at that in a weeks time, when all this RC craziness is behind us19:18
openstackgerritMonty Taylor proposed openstack-infra/system-config: Add a clouds.yaml file in anticipation of shade  https://review.openstack.org/16860119:19
mordredstevemar: ++19:19
mordredclarkb: I updated the commit message on the base one to indicate it's not _just_ about nodepool19:20
*** Sukhdev has quit IRC19:20
mordredclarkb: (fwiw, I'll also want to install one of these on puppetmaster but with a different set of clouds in )19:20
clarkbmordred: thanks, looking19:20
*** esheffield_ has quit IRC19:20
*** notnownikki has quit IRC19:21
*** SumitNaiksatam has quit IRC19:21
*** SumitNaiksatam has joined #openstack-infra19:21
openstackgerritMonty Taylor proposed openstack-infra/system-config: Switch to consuming cloud auth from clouds.yaml  https://review.openstack.org/16860219:21
clarkbmordred: 168602 needs a reparenting now but 168601 helps quite a bit19:21
clarkbbeat me to it :)19:22
*** dhritishikhar_ has quit IRC19:22
*** SumitNaiksatam has quit IRC19:23
*** Guest65061 has quit IRC19:23
clarkbmordred: stevemar one suggestion about clouds.yaml. the region_name should be a yaml list rather than csv19:23
*** rkukura_ has joined #openstack-infra19:24
*** rkukura has quit IRC19:25
*** rkukura_ is now known as rkukura19:25
clarkbmordred: stevemar and the expectation is that service type and name and auth url are all known via os-client-config?19:26
mordredclarkb, stevemar: that's a good idea19:26
mordredclarkb: yes. they all are19:26
mordredclarkb: or can be19:26
clarkbcool19:26
mordredthey are characteristics of a cloud19:26
mordredclarkb: btw - I'm fixing a review comment from ianw on the nodepool side of adding shade support ... so if you're reviewing it I'm about to push up new patches19:27
mordred(he suggested adding validation for the new config option, which I agree with)19:27
clarkbI haven't gotten that far yet but thanks for the heads up19:27
stevemarmordred, does each cloud entry need a 'cloud' value?19:28
mordredstevemar: in nodepool.yaml or clouds.yaml ?19:28
mordredah - clouds.yaml - it does not need one, no19:28
stevemarlooking at the example in the README19:28
mordredit's an optional shorthand to specify a known cloud19:28
stevemarah okay19:29
stevemaris there a list of valid values?19:29
mordredso if you're using rackspace, you can say "cloud: rackspace" and you'll get some default values19:29
stevemargotcha19:29
mordredthe list is "hp,rackspace" right now19:29
mordredalthough I'm going to add dreamhost soon - we should update the README19:29
stevemarmakes sense19:29
openstackgerritMerged openstack-infra/project-config: Create core groups for main fuel-* projects  https://review.openstack.org/16818219:29
*** Somay has joined #openstack-infra19:29
mordredclarkb: oh - btw - I explained in the nodepool series in the commit message that it was optional :)19:30
*** cdent has quit IRC19:30
mordredclarkb: so I apparently knew you needed to know that - but just added it a different commit :)19:30
clarkbnibalizer: can you confirm my comment on 168572? it sa puppetism19:30
*** adalbas has quit IRC19:31
* nibalizer looks19:31
*** carl_baldwin has joined #openstack-infra19:32
clarkbdanke19:32
openstackgerritMatthew Treinish proposed openstack-infra/project-config: Add os-testr repo  https://review.openstack.org/16724319:33
openstackgerritMonty Taylor proposed openstack-infra/nodepool: Pass TaskManager in to shade  https://review.openstack.org/16866319:35
openstackgerritMonty Taylor proposed openstack-infra/nodepool: Use shade for image uploads  https://review.openstack.org/16863319:35
openstackgerritMonty Taylor proposed openstack-infra/nodepool: Use shade to get the client objects  https://review.openstack.org/16860319:35
openstackgerritMonty Taylor proposed openstack-infra/nodepool: Use shade API for flavors  https://review.openstack.org/16863419:35
clarkbnibalizer: basically undef like that will override a default right?19:35
mordredclarkb: I fleshed out the discussion of opt-in in the nodepool support for it there19:35
nibalizerclarkb: yea if you pass in undef... wtf happens idunno i think you get to pick up the default19:35
nibalizeranyways im gonna -119:35
nibalizerfor kinda related reasons19:35
nibalizermore testing needed :P19:36
clarkbmordred: thanks, going to continue reviewing changes after lunch19:37
clarkbbut EHUNGRY now19:37
*** ivar-lazzaro has joined #openstack-infra19:37
mordredclarkb: cool - I actually made a nodepool-dib etherpad19:37
mordredclarkb: to help deal with teh current onslaught19:37
mordredclarkb: https://etherpad.openstack.org/p/dib-nodepool-merges19:37
mordredjeblair, fungi, pleia2, jhesketh, SergeyLukjanov: ^^19:37
*** carl_baldwin has quit IRC19:39
annegentlehey infra mordred jeblair etc, is it generally correct that the big tent is now open?19:40
greghaynesmordred: nice19:40
*** rkukura has quit IRC19:40
annegentlewhat I mean is, can all projects now use docs.openstack.org for docs, using our known frameworks.19:40
annegentleI can bring up the topic at the TC meeting, but looking for "generally, this is what we all think happens now"19:41
*** timcline_ has quit IRC19:41
AJaeger_annegentle: right now I only let projects throught that are in the openstack namespace19:42
annegentleAJaeger_: yes, was thinking that's our new line, what is the timing then? "by kilo"?19:42
annegentleAJaeger_: I mean, I think I know, but wanted to vet with others :)19:42
AJaeger_annegentle: magnum for example was just approved by the TC and therefore can use docs.openstack.org IMHO.19:42
annegentleAJaeger_: yep.19:43
annegentleAJaeger_: same for murano19:43
AJaeger_yep19:43
annegentleAJaeger_: ok19:43
*** claudiupopa has left #openstack-infra19:43
AJaeger_I'm not aware of a timeline - if the TC approves, they can use it is my practical motto ;)19:43
annegentleAJaeger_: :)19:44
*** otter768 has joined #openstack-infra19:45
*** adalbas has joined #openstack-infra19:45
AJaeger_annegentle: yeah, the big tent needs some documentation discussion...19:46
annegentleAJaeger_: I'd like to write up guidelines. We can continue in #openstack-docs. But it will help me think about data-driven tags for docs.19:47
annegentleAJaeger_: like "has CLI Ref, check" "Is in config-ref, check" "is in Install Guide, nope"19:47
annegentleand so on19:47
annegentleseems data driven anyway19:47
AJaeger_yep19:47
AJaeger_let's discuss this in #openstack-docs - but timelines sounds like a TC discussion19:48
nibalizerclarkb: so i tested it and you were wrong (but could easily have been right)19:49
*** otter768 has quit IRC19:49
clarkbnibalizer the default is used?19:50
clarkbI dont even follow how that is sane19:50
clarkbwe have specifically overridden the default with undef19:50
anteayaannegentle: do you have any stipulations about what projects use docs.o.o?19:50
clarkbbyt the default is applied19:50
*** mpavone has joined #openstack-infra19:50
nibalizer!shrug19:51
openstacknibalizer: Error: "shrug" is not a valid command.19:51
anteayaannegentle: in my mind that is your territory and would be telling me what you want to see19:51
clarkbha19:51
crinklepassing param => undef is the same as not setting the param, so the default is used19:51
*** rkukura has joined #openstack-infra19:51
nibalizerclarkb: fwiw i totes see where you're comming from19:51
annegentleanteaya: what we're moving towards is "in the github namespace qualifies you for docs.openstack.org publishing privileges" I suppose? That's sort of what I want to talk through. I'm envisioning the PTL sets quality bars, etc, but is there any legal consideration I should ask the Foundation about? Hm.19:52
nibalizerthis is how you end up passing things like 'EMPTY VALUE' into things19:52
mordredanteaya, annegentle: my understanding is similar to what AJaeger_ said. nothing has changed in terms of supporting stackforge projects, so I would not expect a change in terms of stackforge projects using docs.o.o. what has changed is that the barrier to being in the openstack namespace is much lower19:52
clarkbI mean if I pass None as a kwarg in python I get None19:52
clarkbI dont get $random default19:52
AJaeger_annegentle: for formal defintion, use "git.openstack" instead of github ;)19:52
anteayaannegentle: I'm assuming you meant in the openstack namespace19:53
anteayaannegentle: my point is that you are the one managing what is on docs.o.o in terms of usage and load19:53
crinkleclarkb: None is different from undef19:53
annegentleAJaeger_:woops, yes of course :)19:53
anteayaannegentle: so if you feel you need any qualifiers to what uses it, I'd like to hear your thoughts19:54
annegentleanteaya: but the infra team has done a fantastic job of "self-service" so docs should follow.19:54
anteayaannegentle: if that is your call, I'm fine with that19:54
*** carl_baldwin_ has joined #openstack-infra19:54
AJaeger_annegentle: if you want infra to not approve publishing to docs.o.o, please give us a policy/guidelines19:55
anteayaannegentle: my point is I don't think there is a default decision here, it is whatever you say works for you19:55
anteayaannegentle: in the absence of any guidelines from you, any project in the openstack namespace can publish to docs.o.o19:56
clarkbcrinkle they are both the absence of value values19:56
anteayaannegentle: but anything from you that overrides that supercedes that19:56
annegentleanteaya: ok, that's helpful, thanks!19:57
* AJaeger_ agrees with anteaya19:57
anteayaannegentle: thanks so if we need to do anything other than what we are doing now, be sure to let us know19:58
crinkleclarkb: you're right, my mistake. puppet just has different behavior than python19:58
crinkleclarkb: they're changing the behavior of undef in puppet 4 since this is obviously confusing19:58
*** Swami has quit IRC19:59
*** dprince has quit IRC19:59
clarkbcrinkle thats good. maybe we should respect puppet 4s change for future proofing then?19:59
clarkbnibalizer ^19:59
*** AJaeger_ has quit IRC20:00
*** julim has quit IRC20:02
*** annegentle has quit IRC20:04
*** peristeri has quit IRC20:04
nibalizeruh, i think thats a bigger conversation kinda?20:05
nibalizerI'd like to get some minimal commitment that we want to upgrade to puppet 4 before choming off a bunch of work to make infra puppet4 go20:05
bookwarAJaeger: hi, thank you for merging this thing https://review.openstack.org/168182 I am worried about how gerrit changes are applied. To not block current development process, while we are populating these new groups, we need fuel-core to be added to all groups which were created by this patch20:07
fungibookwar: i can do that once puppet finishes updating gerrit to create them20:08
anteayanibalizer: what are the advantages/disadvantages of moving to puppet 4?20:09
mordrednibalizer: I'd like to get on to puppet apply before we start puppet 4-ing20:09
bookwarfungi: thanks, so i'll wait then20:10
mordrednibalizer: because I think that shoudl make it "easier" due to being able to do it piecemeal, yeah?20:10
*** peristeri has joined #openstack-infra20:10
nibalizerwell... first some background, puppet4 isnt out yet20:10
nibalizerhowever, its sorta bundled with puppet 3.7, add future_parser = true to puppet.conf and you get puppet420:11
mordredcool20:11
nibalizermordred: yes puppetapply will make it easier20:11
mordredthat's a nice testing step20:11
nibalizermordred: yea they learned from last time20:11
mordred++20:11
nibalizerthey've had that flag available for like a year, so people have been kicking the tires20:11
nibalizeranteaya: some advantages we see above, its a better language with better handling of undef and basic types20:12
anteayanibalizer: do we need that20:12
nibalizerwe can throw away the 'validate_*' crap that we've had to do, it should be faster20:12
anteayaare we blocked in what we do without having that?20:12
nibalizerthe disadvantage is that its work to upgrade20:12
nibalizeranteaya: not blocked at all20:12
*** mwhudson has quit IRC20:12
nibalizerreally upgrading to puppet4 isn't even on my radar really20:13
nibalizeri have other things i want to fix in infra more20:13
anteayaso yes, remembering the move to puppet 3 was a lot of work, mostly on your shoulders20:13
anteayanibalizer: so given that, I would be on the side of letting you fix the stuff in infra you think is more important20:13
*** carl_baldwin_ is now known as carl_baldwin20:13
*** mwhudson has joined #openstack-infra20:14
*** carl_baldwin has quit IRC20:14
openstackgerritMonty Taylor proposed openstack-infra/shade: Pass socket timeout to swiftclient  https://review.openstack.org/16867220:14
jeblairmordred: i left feedback in the etherpad -- i'd like to either have a real plan for a build infrastructure, or documentation for the manual steps to use the ppa, or both...20:16
jeblairmordred: i think i'm going to -1 167625 based on that, even though i don't think it is something that can actually be fixed in the patch; does that sound okay?20:16
mordredjeblair: yup!20:17
mordredjeblair: do you think I should upload to an infra-core-available PPA so that any infra-core can upload a new pacakge to it?20:17
jeblairmordred: that would probably be a good idea.  it should make the 'document the manual process' easier for now and not be too difficult, right?20:18
mordredyup20:18
mordredjeblair: of of https://launchpad.net/~openstack-ci-core perhaps?20:18
mordredjeblair: also - we shoudl add pleia2 and jhesketh there?20:19
jeblairwow that's dusty20:19
jeblairand yep.20:19
mordredjeblair: where should I document taht? system-config docs?20:20
jeblairmordred: yep, probably nodepool section for now, but i think it would be more fundamental puppet stuff if it becomes required for infra server launching20:21
*** timcline has joined #openstack-infra20:22
mordredjeblair: we can move it if/when it does20:22
*** timcline has quit IRC20:22
jeblair++20:22
*** timcline has joined #openstack-infra20:23
nibalizeretherpad where?20:23
jeblairnibalizer: https://etherpad.openstack.org/p/dib-nodepool-merges20:23
* clarkb does a happy dance https://jenkins07.openstack.org/job/check-tempest-dsvm-aiopcpu-full/29/console ran succussfully using 64bit guest oses20:23
clarkbjogo: ^20:23
clarkband live migration ran20:24
*** annegentle has joined #openstack-infra20:25
*** tiswanso has quit IRC20:25
anteayaclarkb: congratulations20:25
*** dizquierdo has joined #openstack-infra20:26
*** MaxV has joined #openstack-infra20:27
*** samueldmq has quit IRC20:27
*** afazekas has joined #openstack-infra20:27
jeblairclarkb: see followup comment on 16860120:28
*** bswartz has quit IRC20:28
openstackgerritStephanie Miller proposed openstack-infra/system-config: Add infra setup for zanata  https://review.openstack.org/16907420:30
openstackgerritMonty Taylor proposed openstack-infra/system-config: Document the vhd-util ppa  https://review.openstack.org/16907520:31
*** tsg has quit IRC20:31
openstackgerritMonty Taylor proposed openstack-infra/puppet-nodepool: Install additional depends for dib  https://review.openstack.org/16762520:31
mordredjeblair: ^^20:31
mordredclarkb: woot!20:32
*** melwitt has quit IRC20:33
*** melwitt_ has joined #openstack-infra20:33
*** afazekas has quit IRC20:34
*** ociuhandu has quit IRC20:34
*** tsg has joined #openstack-infra20:35
fungibookwar: i've added fuel-core as an included group of the 10 groups created by that change... fuel-astute-core, fuel-devops-core, fuel-docs-core, fuel-library-core, fuel-main-core, fuel-ostf-core, fuel-plugins-core, fuel-qa-core, fuel-stats-core and fuel-web-core20:37
*** afazekas has joined #openstack-infra20:37
bookwarfungi: exactly, thanks20:39
*** mwagner_lap has quit IRC20:40
fungiyou're welcome!20:41
*** annegentle has quit IRC20:41
*** carl_baldwin has joined #openstack-infra20:42
*** mrda-away is now known as mrda20:43
clarkbdo you guys see my edits on the etherpad as being in a different font? I have no idea why that is happening20:43
*** afazekas has quit IRC20:44
jeblairclarkb: nope, same font, different color20:44
*** dizquierdo has quit IRC20:45
*** e0ne has quit IRC20:46
*** Sukhdev has joined #openstack-infra20:46
mordredclarkb: maybe it's a portland thing?20:47
anteayaclarkb: I see your edits in the same font as the rest of the etherpad in a lovely hot pink20:48
clarkbcould be, my fonts are cool before I even know about it20:48
mordredclarkb: :)20:48
fungilooks like the same font, but with glasses and a handlebar mustache20:48
*** Rockyg has quit IRC20:48
anteayawho is neon turquoise?20:50
anteayamordred: that you?20:50
mordredanteaya: yeah20:50
anteayamust be you20:50
anteayanice20:50
*** annegentle has joined #openstack-infra20:52
dansmithclarkb: awesome20:54
clarkbmordred: https://review.openstack.org/#/c/168673/2 has a comment for clarification which you may not see since its over on swiftlcient20:54
*** rfolco has quit IRC20:54
*** adalbas has quit IRC20:56
openstackgerritStephanie Miller proposed openstack-infra/system-config: Miscellaneous linter cleanups  https://review.openstack.org/16908520:56
*** sarob has joined #openstack-infra20:58
*** signed8bit has joined #openstack-infra20:58
mordredclarkb: thanks20:58
*** yamamoto has joined #openstack-infra20:58
mordredclarkb: and he's right, right? it is a socket read timeout20:58
clarkbmordred: yup I think so20:59
clarkbmordred: also given the two dib changes you linked, do I need to resurrect my nodepool change to support multiple output formats20:59
*** amotoki has joined #openstack-infra20:59
clarkbmordred: I *think* it should mostly just work since shade is handling uploads and dib will use vhd-util20:59
mordredclarkb: yah - I believe so20:59
mordredclarkb: to both20:59
clarkbok will clean that up soon21:00
*** kgiusti has quit IRC21:00
clarkbI wasn't sure if nodepool would have to grow the vhd-util stuff internally21:00
mordredclarkb: yeah - thankfully not21:00
*** melwitt_ has quit IRC21:01
*** melwitt has joined #openstack-infra21:01
*** dkliban is now known as dkliban_afk21:01
*** melwitt has quit IRC21:02
*** ociuhandu has joined #openstack-infra21:02
*** tsg has quit IRC21:04
*** radez is now known as radez_g0n321:06
*** afazekas has joined #openstack-infra21:06
clarkbmordred: greghaynes question for you on https://review.openstack.org/#/c/156433/ too21:07
fungimordred: just out of curiosity because i haven't looked very closely at it yet... vhd-util is basically a fork of the xen codebase?21:08
*** melwitt has joined #openstack-infra21:09
mordredfungi: maybe?21:09
clarkbI think its the xen source with one patch over the top21:09
mordredyah21:09
clarkbhttp://blogs.citrix.com/2012/10/04/convert-a-raw-image-to-xenserver-vhd/21:10
fungimordred: is the end-game for that utility to get it shipped as part of xen itself?21:10
clarkbfungi: its already shipped by xen21:10
clarkbfungi: the problem is the patch that is getting applied has not gotten merged. Its not the greatest code21:11
fungiokay, so what we really need is for the distro xen packages to split that utility out?21:11
*** mpavone has quit IRC21:11
*** dustins has quit IRC21:11
clarkbfungi: and we need upstream (whether xen or distro) to either take the patch as is or fix it21:11
mordredgreghaynes: ImportError: No module named '__builtin__' :)21:11
clarkbI personally think it would be nice to not use -t 1 and -t2 to describe image types21:11
mordredfungi: yes, split the utility out, and also apply that patch21:12
clarkbor have to manually do two passes for an image conversion21:12
mordredI dont' think we shoudl have to convert our image at all21:12
clarkbthat too :)21:12
fungiokay, but there is some endgame where we're (openstack infra) not maintaining some heavyweight fork of the entire xen codebase just to get access to one utility shade needs21:12
mordredI think the glance API should always accept a RAW upload21:12
mordredfungi: I do not believe that I believe in that endgame, but I do think there is at least a theory of that endgame21:13
greghaynesmordred: orly21:13
*** isviridov has quit IRC21:13
clarkbfungi: I don't think anyone is working on that. The blog above is ancient21:13
*** sarob has quit IRC21:13
*** mpavone has joined #openstack-infra21:13
mordredgreghaynes: but I'm goign to upload a new version in response to slagle's comment - so don't fix your thing yet21:13
*** tsg has joined #openstack-infra21:13
clarkbgreghaynes: yes, glance should do glancy things and make me not care about hypervisor21:13
* fungi sighs at... just... everyone and everything in the world21:13
greghaynesclarkb: hah, sorry, that was re: the no module named builtin21:14
clarkbgreghaynes: oh derp21:14
jeblairfungi, clarkb, mordred: i actually think it would be a fun weekend project to reimplement the vhd conversion from scratch in python.  or rust.  can't be that hard, right?  :)21:15
clarkbI know little about image formats but vhd has been nothing but trouble. It would also be awesome if Xen/hyperv supported image formats that were compressed and all that fun stuff21:15
greghaynesclarkb: yea, I think your comment will work about not doing the extra copy, need to test21:15
jeblairotoh, it is spring, so i won't spend my weekends doing that.21:15
*** tnovacik has quit IRC21:15
clarkbgreghaynes: my concern there is the -i and -o being the same21:16
greghaynesclarkb: oh, well thats fine21:16
clarkbgreghaynes: does vhd-util use a temporary file name then rename when its done?21:16
clarkbgreghaynes: otherwise it likely isn't fine21:16
*** ldnunes has quit IRC21:16
fungijeblair: yeah, seems like it would make a fine lib for glance (or even nova) to load, and include a cli endpoint so it can also be used as a general conversion tool21:17
clarkbespecially if in the course of a conversion you can write ahead of your read pointer21:17
greghaynesclarkb: I believe so, which is why the .bak needs to be rm'd21:17
greghaynesclarkb: but that is tested an dworks21:17
clarkbgreghaynes: where does intermediate.bak come from? that chunk of code just reads as buggy21:18
*** sdake has joined #openstack-infra21:18
*** boris-42 has quit IRC21:18
greghaynesclarkb: the vhd-util conversion creates it21:18
fungithat blktap2 patch doesn't look terrible. i wonder why xen upstream is allergic21:18
clarkbgreghaynes: does it create it if -i and -o are different? if not then I extra vote for different -i and -o21:18
*** afazekas has quit IRC21:18
fungii mean, more than half the patch is comments21:19
marunneutron is wedged until this merges: https://review.openstack.org/#/c/169092/21:19
mtreinishjeblair, clarkb, fungi: I was wondering if there was an interest in setting up a submodule repo for all the puppet things21:19
maruncan we get it bumped if/when a reset occurs?21:19
fungisome are even usefully commenting the not-yet-commented code which was already there21:19
mtreinishbecause a common problem I have now is finding things post split21:19
clarkb fungi its incredibly user unfriendly21:20
mordredclarkb, fungi, jeblair: in theory, qemu-image _should_ support doing this but does not21:20
mordredso honestly, if someone were going to hack on things, just fixing qemu-image would be the best chocie21:20
*** mpavone has quit IRC21:20
greghaynesclarkb: so are you worried that theres a potential issue where it works sometimes and not others? Its a pretty obtuse tool and I have no idea what the internals that are going on are21:20
clarkbmtreinish we have a module installer in system-config21:20
jeblairmarun: did the devstack up/down job not catch that?  is that fixable?21:20
greghaynesthat setup does work for booting rax images though21:21
fungiclarkb: oh, so it's less about code quality and more that it's just a terrible idea to implement it that way?21:21
clarkbgreghaynes yes exactly and the dib change reads as wrong21:21
clarkbfungi yes :)21:21
mordredclarkb, greghaynes: also, the raw to first .vhd happens in place - the second conversion makes a backup file :)21:21
*** SumitNaiksatam has joined #openstack-infra21:21
*** annegentle has quit IRC21:21
marunjeblair: it's only the neutron functional job that's broken, and there is nothing stopping ds from merging breaking fixes21:21
greghaynesok, so if it happens in place why is there a -o at all?21:21
greghaynesclarkb: what reads as wrong21:21
mordredgreghaynes: it does the move21:21
*** sdake_ has quit IRC21:21
clarkbfungi: see current discussion about inputs and outputs and backup files21:21
marunjeblair: sdague kindly tagged a release that we could pin to but I hadn't done so yet.  will make it a priority now21:21
mordredgreghaynes: I think21:22
mordredgreghaynes: it's a terrible tool21:22
clarkbfungi: also the -s and -t flags take integer args which is lolwut21:22
jeblairmarun, dtroyer: yeah, but i thought there was a job on devstack to test the parts used by functional tests21:22
marunsdague: could you add a kilo3 tag?21:22
mordredclarkb: it's amazingly bad21:22
jeblairmarun: my concern is that seems not to have worked21:22
clarkbgreghaynes: deleting a file you never create (the .bak), using an input and output that are the same21:22
jeblairmarun: tagged a release of what?21:22
clarkbgreghaynes: those are both smells in shell scripts21:22
marunjeblair: they might test the parts that are used, but in this case it was a 'source lib/neutron' that broke21:22
fungiclarkb: agreed21:22
marunjeblair: The intent was to tag a milestone release of devstack we could pin to21:23
mtreinishclarkb: it was more for grepping and keeping things up to date all at once, not for necessarily running things21:23
jeblairmarun: how would you pin to it?  i don't think that does (or should) work...21:23
clarkbmtreinish: it should do that21:23
marunjeblair: we'd checkout the tag in the configure script before we sourced anything21:24
greghaynesclarkb: so you want me to use different outputs and not rm the .bak?21:24
jeblairmarun: please don't do that, you will defeat cross-project testing21:24
marunjeblair: ??21:24
greghaynessounds like the tool does an in place modify and then maybe moves21:24
clarkbgreghaynes: if possible. If the .bak is created even when inputs and outputs are different then maybe we need a comment to explain the sillyness21:24
greghaynesso its all the same21:24
marunjeblair: we're only talking about pinning devstack for a script that installs os deps21:24
greghaynessure21:24
marunjeblair: it doesn't have anything to do with cross-project21:24
marunhttps://github.com/openstack/neutron/blob/master/tools/configure_for_func_testing.sh21:25
jeblairmarun: you're using devstack to set all of this up though, right?21:25
marunjeblair: I'm using devstack functions, yes.21:25
marunjeblair: I don't invoke stack.sh21:25
mtreinishclarkb: sure, but it's different than my normal git workflow which is kinda my issue21:26
clarkbgreghaynes: I prefer code to be readable without knowing silly magic in broken tools21:26
clarkbleast surprise etc21:26
greghaynesclarkb: sure, unfortunately were using a magic black box of a tool though21:26
clarkbya :/21:26
mordredgreghaynes: also - I'm out of that dib change if you want the lock back to work on that test21:26
jeblairmarun: so what should have happened is that the change to devstack should not have merged because it broke a job that depended on devstack.  i believe the best way to fix that is to ensure devstack has adequate testing of this...21:27
anteayaclarkb: I'm curious what patch is under discussion, I've looked through the backscroll and if it is there I haven't seen it yet21:27
greghaynesmordred: the vhd or config drive process one?21:27
greghaynesor both21:27
*** ibiris is now known as ibiris_away21:28
clarkbanteaya: https://review.openstack.org/#/c/156433/10 that one21:28
anteayathanks21:28
marunjeblair: testing of what? the fix is s+lib/neutron+lib/neutron-legacy+21:28
clarkbmtreinish: typically we have resisted submodules because people trip on them always21:28
anteayathis created the fungi sigh, a memorable event21:28
mordredgreghaynes: the config drive one21:28
mordredgreghaynes: both actuall - I'm done mucking with both21:28
mordredgreghaynes: whichever one is broken21:28
greghaynesok, I think both ;)21:28
marunjeblair: how do we track something like the name of an import across everyone that imports it?21:28
jeblairmarun: part of devstack's api is now "lib/neutron" -- it broke that api because no jobs running on devstack test that.  i think that should be corrected.21:28
mordredhehe21:29
marunjeblair: I guess I'm not sure that the devstack folks agreed to keep it stable.21:29
jeblairmarun: eek.21:29
mordredcan we reverse that trend?21:29
mordredI'm a little tired of people thinking they can opt in to supporting an api21:29
marunjeblair: I asked to have a setup-only neutron job added to the devstack gate pipeline to prevent this, but that was deemed unacceptable.21:29
mordredif it's a detail, mark it that way21:29
mordredin python, we use leading _'s for those things21:30
*** boris-42 has joined #openstack-infra21:30
jeblairmarun: i thought there was one21:30
marunjeblair: the compromise was to tag a stable devstack release and pin21:30
marunjeblair: and it's my fault for not pinning21:30
marunjeblair: a pin isn't intended to be a get-out-of-jail-free card.21:30
marunjeblair: it would be my responsibility to keep current21:30
clarkbmtreinish: we do use them for our gerrit repo beause that is how gerrit does plugins and for the submodule track openstack openstack/openstack repo21:30
jeblairmarun: i don't think projects should have pins on devstack21:30
clarkbmtreinish: so in theory its possible but I think in general we would try to avoid it21:31
marunjeblair: and fix breakage as it occurs.  but at least that wouldn't block other developers21:31
marunjeblair: What are you concerns with a pinning-based strategy?21:31
mtreinishclarkb: yeah openstack/openstack was what I was thinking of21:31
jeblairmarun: because _one_ job will have neutron set up in a certain way, and other jobs will have it set up in a different way21:31
marunjeblair: our use of devstack doesn't suggest any kind of stable api21:31
marunjeblair: it's not neutron that' being set up, though.21:32
jeblairmarun: i don't see how you can evolve what devstack does to set up neutron for integration testing separately from what it does for functional testing21:32
marunjeblair: the functional job manages its own fixture21:32
mtreinishclarkb: sure I can understand that, submodules can get really weird. I just miss the one big repo for everything, it worked well for my infrequent puppet contribs21:32
marunjeblair: it's only os deps that the script installs21:32
jeblairmarun: what's the 'source lib/neutron' for then?21:32
marunjeblair: to allow the installation of agent os deps21:32
*** mattfarina has quit IRC21:32
marunjeblair: that's it21:32
marunhttps://github.com/openstack/neutron/blob/master/tools/configure_for_func_testing.sh#L16621:33
jeblairmarun: shouldn't those still evolve together?21:33
jeblairmarun: if you add/change an os dep for integration testing, shouldn't that also be reflected and tested with functional testing?21:33
marunjeblair: sure, but the addition that tries to add the os dep can be responsible for updating the pin21:33
*** dkranz has quit IRC21:34
*** yamahata has quit IRC21:34
*** carl_baldwin has quit IRC21:34
jeblairmarun: how?  it's a change to devstack, right?21:34
marunjeblair: we shouldn't be merging code without functional tests, and if that requires devstack updates, then the code depending on a newer devstack can't merge until the pin gets updated once devstack has thechange21:34
jeblairmarun: meanwhile, the integration tests are using the new thing while the functional tests are not.21:35
openstackgerritMatt Riedemann proposed openstack-infra/elastic-recheck: Add query for ironic sideways bug 1438415  https://review.openstack.org/16909921:35
openstackbug 1438415 in devstack "Some network tests failing with IndexError" [Undecided,In progress] https://launchpad.net/bugs/143841521:35
marunjeblair: no21:35
marunjeblair: er21:35
marunjeblair: we're talking os deps here...21:35
marunjeblair: So the only change would be 'I'm adding something new'21:35
*** jamielennox|away is now known as jamielennox21:36
marunjeblair: and nothing would be using that new thing until a change to neutron merged21:36
marunjeblair: if we were talking python dep that would be different - the versions might change21:36
*** annegentle has joined #openstack-infra21:36
clarkbgreghaynes: I updated my comment on the vhd-util dib change, does that make sense?21:36
marunjeblair: but for a given distro the dep version is going to be common across all installs21:37
clarkbgreghaynes: unfortunately vhd-util doesn't seem to have a manpage or other documentation I can refer to21:37
greghaynesYep, I am very aware of that fact ;)21:38
jeblairmarun: i guess i'm not as certain that the only changes that will ever affect neutron functional tests in devstack are os dependency additions.  surely there's other stuff going on, right?  configurations of certain things?  if os deps is really all it is, why use devstack at all?21:38
greghaynesIt also wouldnt apply since were using a patched version of it21:38
greghaynesbasically, this is "we found an incantation that seems to work so were doing it"21:38
marunjeblair: service configuration could potentially change21:38
marunjeblair: mysql/pg and rabbit custom and distro-specific fixups21:38
jeblairmarun: wouldn't it be good for that to stay in sync between functional and integration tests?21:39
marunjeblair: I don't think it's as critical as not blocking developers21:39
marunjeblair: the fact that mysql or rabbit behave a bit differently is unlikely to break tests21:39
jeblairmarun: if you are saying that my suggestion is to block developers, we have a big communication problem here21:39
jeblairmarun: you _are_ blocked now, and i'm trying to help you fix it.21:39
marunjeblair: I guess I'm not seeing a good alternative to pinning that doesn't involve more work for little gain21:40
clarkbgreghaynes: magic21:40
marunjeblair: in an ideal world we have devstack-lib that knows how to install things in distro-specific and openstack-specific ways21:40
jeblairmarun: simple -- we fix the devstack job that is _supposed_ to test this so that it does, and if it turns out it isn't really supposed to test it, then we add the job you suggested earlier.21:40
jeblairmarun: because we need to accept that if devstack is going to be used by functional tests, it needs to have a tested api21:41
jeblairdtroyer, mtreinish: so what can we do to the devstack up/down job to make sure this part of devstack's api is tested in the future?21:41
mtreinishjeblair: there is a test job which does stack than unstack21:42
jeblairmtreinish: so it doesn't actually use the part of the devstack api that a functional test job might use?21:42
mtreinishjeblair: http://logs.openstack.org/07/168407/7/check/check-devstack-dsvm-updown/385a90c/console.html21:43
dtroyerjeblair: the project include file are not part of any imagined API that DevStack may or may not have.  The only thing that has 'permission' to use them directly is Grenade and those two pojects often have to have coordinated changes to make things work.21:43
*** timcline has quit IRC21:43
greghaynesclarkb: also, this just makes me want to make jobs to test all the image creation formats more21:43
greghaynesclarkb: is adding a custom ppa for a job like that doable?21:44
jeblairmarun: i read that as dtroyer saying that functional test jobs should not use those libs21:44
marunjeblair: hah21:44
marunjeblair: well, I could stop using devstack and roll my own21:44
dtroyerwhat marun is doing is playing with sharp objects and he got cut.  if there needs to be some devstack-lib-like api, then we need to define those, until then I do not accept that anything in devstack is open for other use just because it exists21:44
marunjeblair: or I could use that time to do useful work and pin21:44
jeblairmarun: let's actually discuss that seriously for a minute21:44
jeblairmarun: what are you actually using from devstack?  os dependencies i got, anything else?21:45
dtroyerif installing os deps beyond what we pulled out into tools is required, then that needs to be defined and pulled out somewhere21:45
marunjeblair: no.  os deps is all21:45
*** mrmartin has quit IRC21:45
*** ajmiller_ has joined #openstack-infra21:45
marunjeblair: not just package installation, but post-installation fixup, sometimes distro-specific21:45
*** mriedem is now known as mriedem_away21:45
dtroyerthat's one reason I keep moving things into tools.  those are available21:45
mordredI believe the bindep work should help this, no?21:45
jeblairmordred: yeah, that's where i'm going with this21:45
mordredjeblair: sorry, I'll butt out :)21:46
jeblairmordred: but trying to figure out if it's feasible21:46
*** otter768 has joined #openstack-infra21:46
jeblairmordred: no problem, please stay butted in :)21:46
*** jamielennox is now known as jamielennox|away21:46
jeblairmarun: what about mysql installation/config/etc?  or rabbit?  or any other service startup and configuration?21:46
marunjeblair: we do the same thing devstack does. package install, distro and openstack-specific post-install fixup/config21:46
marunjeblair: it's all there in the script21:47
marunrpc: https://github.com/openstack/neutron/blob/master/tools/configure_for_func_testing.sh#L9621:47
marundb: https://github.com/openstack/neutron/blob/master/tools/configure_for_func_testing.sh#L11121:47
*** MaxV has quit IRC21:47
marunbase deps: https://github.com/openstack/neutron/blob/master/tools/configure_for_func_testing.sh#L8221:47
marunagent deps: https://github.com/openstack/neutron/blob/master/tools/configure_for_func_testing.sh#L16321:47
openstackgerritMerged openstack-infra/elastic-recheck: Add query for ironic sideways bug 1438415  https://review.openstack.org/16909921:48
openstackbug 1438415 in devstack "Some network tests failing with IndexError" [Undecided,In progress] https://launchpad.net/bugs/143841521:48
marunit would be great to simply have a function call for each of these21:48
marunand have the resulting api supported21:48
*** ajmiller has quit IRC21:48
jeblairmarun, dtroyer: it looks like there's some copying from devstack into there, but also invocation of other things from devstack/lib/*  (not just neutron).  is that correct?21:49
dtroyerthere is, and I didn't know it was going that far.  I would suggest that is dangerous…21:49
marunjeblair: correct. we need to install db and rpc21:49
jeblairdtroyer, marun: so could devstack evolve an API that does things like "install_and_setup_db" ...?  and would that be useful?21:50
*** julim has joined #openstack-infra21:50
marunjeblair: absolutely. none of what I'm doing couldn't be done better in devstack.21:50
dtroyerit could.  I'm accepting gifts of days with which to do so…21:51
*** otter768 has quit IRC21:51
* anteaya hands dtroyer a Friday21:51
dtroyeralthough it'll be pretty low on the priority list, ie we didn't even talk about this last week when sorting out the priorities21:51
marunwhich is why I went ahead and just did it21:52
marunand I'm still of the opinion that pinning is the best way to do this until more projects need the capability21:52
marunneutron is it for now as far as I know21:52
jeblairmarun, dtroyer: so how about we start to define that api, and we set up a job that makes sure it is all tested?  perhaps what's in neutron could be moved into devstack until the neutron func setup script is just a series of devstack api invocations?21:52
*** jamielennox|away is now known as jamielennox21:53
jeblairmarun, mordred: to finish out the thought from ealier, because of the setup involved, i don't think bindep is the answer here; it would be a good one if we only cared about installing deps, but there's actual configuration going on which i think everyone at least agrees ideally should be in devstack.21:54
dtroyerI'll think about it after venvs are implemented and backported and a sensible neutron config is running as our default.  at the current rate we'll talk about it in tokyo21:54
mordredjeblair: nod21:55
*** aysyd has quit IRC21:55
marunjeblair: I'd be happy to have things simpler. It doesn't sound like there are the cycles ot make it happen soon, though.21:56
*** rbradfor has joined #openstack-infra21:56
jeblairmarun: okay.  so under the circumstances, i think either pinning or just copying the bits of devstack you need into your repo are the options on the table.  devstack has no public api for what you want to do, so testing that wouldn't seem to be the solution.21:56
openstackgerritAaron Rosen proposed openstack-infra/devstack-gate: Allow SERVICE_TIMEOUT to be configured  https://review.openstack.org/16910321:56
jheskethMorning21:57
jeblairmarun: honestly, i figure you've done much of the work already, so i'm happy to try to volunteer you for shaping at least part of that api, but it doesn't sound like there'd be a lot of bandwidth on the receiving side, so i won't push that.  :)21:57
anteayamorning jhesketh21:57
marunjeblair: I'd be happy to help when the cycles are available.21:57
jeblairmarun, dtroyer: our thought from a while back was "let's see a project implement functional testing and use that as a template for the rest"21:57
jeblairmarun, dtroyer: unfortunately, it seems like what we ended up with is not a good template21:58
jeblairso i feel that we should not accept new functional jobs that involve devstack bits21:58
marunjeblair: :/21:58
marunjeblair: I don't think that's a good solution21:58
jeblairi don't either21:59
anteayajeblair: oh, how should folks set up functional jobs?21:59
anteayawhat should we give them as guidance?21:59
clarkbI'm a fan of the zuul/nodepool functional teting model though it probably doesn't map onto testing things with many system requirements very well22:00
jeblairanteaya: i think the thing we have discovered is that functional jobs can't use certain parts of devstack.22:00
clarkb(they basically set everything up in test)22:00
jeblairanteaya: and it will be hard for us to determine whether a given job does so or not22:00
marunjeblair: I don't see why what we're doing now isn't viable with pinning22:00
anteayajeblair: yes, I am reading that22:00
marunjeblair: until an api comes along22:00
anteayajeblair: ah22:00
marunjeblair: I want to get stuff done, not block until we have some ideal22:00
jeblairmarun: i don't think i'm suggesting anything that will inhibit your work?22:01
jeblairmarun: i'm suggesting that the _next_ functional job that comes along should take what we've learned from this attempt and come up with a better one22:01
marunjeblair: fair enough22:02
*** unicell has quit IRC22:02
*** unicell has joined #openstack-infra22:02
marunjeblair: I think fungi was actually working towards similar ends in his work to allow unit tests to run on devstack nodes22:02
anteayamarun: yes I too am reading that a _new_ functional job needs more scrutinty22:02
jeblairmarun: yeah, that's the bindep thing i was talking about earlier22:02
*** nelsnelson has quit IRC22:02
marunjeblair: yeah, but as you say, bindep doesn't manage distro-specific fixup or openstack-specific post-install config22:03
*** YorikSar has quit IRC22:03
jeblairyep22:03
marunjeblair: I'd be happy to see it take that on, frankly.22:03
marunjeblair: devstack could orchestrate and bindep could manage package install22:03
jeblairmarun: oh yeah, we're definitely doing that :)22:03
* fungi is working on it as we speak in fact. cross-checking package sets on held workers of divverent types22:03
*** hashar has quit IRC22:04
fungitrying to make sure https://review.openstack.org/164447 covers all our current bases22:04
anteayajeblair fungi clarkb sorry to change the topic but on my action item to come up with a post to the infra list about election tooling, I'm inclined to delay until after this election season is over22:04
anteayaso about 4 weeks, so as not to add to the confusion that I initially was trying to avoid22:04
marunfungi: any thoughts about expanding the scope of bindep?22:05
anteayathoughts?22:05
jeblairanteaya: i don't see a huge reason not to start the thought process now, but i'm happy either way.  :)22:05
fungimarun: er, depends on what scope expansion you're considering. it's aimed at being a simple tool to parse a list of packages and let you know what you're missing22:05
fungimarun: with the expectation that you can feed the result into apt-get install or yum install or whatever22:06
anteayajeblair: you don't think a thread on infra with a topic of elections might confuse folks?22:06
jeblairanteaya: only usefully :)22:07
anteayajeblair: ha ha ha22:07
anteayawell here's hoping22:07
marunfungi: :/22:08
jeblairmarun: to go waaay back to your original question -- i don't think we need to promote that change because it only affects neutron functional jobs which are only on the neutron repo, so any job behind that change in the gate queue will use it anyway.  if you want validate jobs in check with that in place without waiting for it to merge, you can set "Depends-On: I5eb4b4052da4b0db128feb42feae50a8bc59f373"22:08
fungijeblair: marun: also it's already in the gate with only two changes ahead of it in its shared change queue anyway22:09
jeblairand that22:09
marunyeah, looking good22:09
jeblairdtroyer: is there someplace where devstack's "api" is described?  if not, it would probably be good to have that in writing, particularly what you think is acceptable for a functional test job to use.22:10
*** carl_baldwin has joined #openstack-infra22:10
*** amotoki has quit IRC22:10
marunfungi: so devstack/shell scripting/puppet still get to do post-install stuff...22:11
*** miqui has quit IRC22:11
marunfungi: I think the duplication of effort and lack of consumability of this status quo is unfortunate.22:11
*** paul-- has quit IRC22:11
openstackgerritMonty Taylor proposed openstack-infra/shade: Migrate API calls to task management  https://review.openstack.org/16866522:11
openstackgerritMonty Taylor proposed openstack-infra/shade: Add task management framework  https://review.openstack.org/16866422:11
openstackgerritMonty Taylor proposed openstack-infra/shade: Fix a use of in where it should be equality  https://review.openstack.org/16910522:11
*** paul-- has joined #openstack-infra22:12
dtroyerjeblair: not really…  other than Grenade, the only things meant for external use is functions* (they are supposed to stand alone) and tools/*22:12
marunwhich disallows installation of rpc db22:12
dtroyeradd inc/* as we're breaking up functions-common and adding tests22:12
marunand anything service-specific22:13
fungimarun: well, i'm not opposed to a tool scoped to perform configuration of things, but that sounds like something better decomposed out of devstack. i picked up bindep because it already existed, just needed a bit of fleshing out, but i think it's trying to be a traditional unix "do one thing and do it well" sort of utility22:13
jeblairdtroyer: can you write that down somewhere?22:13
dtroyerthings like the rpc and databse that marun needs need to be pulled out somehow if they are to be generally used, again, other than Grenade22:13
dtroyerjeblair: I'll add it to the list22:13
jeblairmarun: i agree, but i think we need to address that by resetting expectations around the proper tools.  bindep isn't the place for that, but devstack or something like it is.  in fact, a benefit of keeping bindep simple is that we can use it in devstack, unit test jobs, and functional test jobs.  all of which need "some packages installed".22:13
marunjeblair: the hard part is not installing a list of deps22:14
jeblairon top of that, _some_ (but not all) of those need configuration done.22:14
jeblairmarun: i agree ^ that is the hard part22:14
*** dboik_ has joined #openstack-infra22:15
anteayajeblair fungi clarkb tristanC mordred anyone else who cares about elections: https://etherpad.openstack.org/p/XU4qRouXIH22:15
jeblairmarun: and i don't have a solution to that right now other than devstack.  so we either need to work with dtroyer to evolve it there (once he gets over the shock we're putting him through now, and we have time to plan this out properly) or decide that a different tool would be better.  and hey, maybe devstack uses that.22:15
anteayanote not pinging pleia2 as she is away this week22:15
*** asselin has joined #openstack-infra22:15
*** yamamoto has quit IRC22:15
anteayamordred: I recognize your colour now22:16
marunjeblair: So long as we can run functional tests I'm not wedded to how we configure the environment.22:16
fungijeblair: marun: for example, something to take on the tasks performed by the .*sql-{db,prep} macros at http://git.openstack.org/cgit/openstack-infra/project-config/tree/jenkins/jobs/macros.yaml#n439 would be helpful22:16
marunjeblair: I don't like the idea of stalling testing effort in other projects because 'devstack isn't ready' though - by that logic we shouldn't have done what we've done.22:16
fungijust looking at things from a decomposability perspective22:16
marunjeblair: and I think making progress > doing things 'right'22:17
marunjeblair: we can always fix things that break.  I'd rather risk breakage than stall.22:17
fungithough sometimes "making progress" really means "accumulating technical debt we can never hope to pay back"22:17
*** dboik has quit IRC22:18
marunfungi: we have to consider that possiblity, but I don't think it makes sense to consider things in that light by default.22:18
jeblairmarun: i understand, but i also don't think that a bunch of other projects asking that devstack tag a new release with a new dep updated, but please also don't break anything that might make it harder to release a new tag for that other project later is actually a scalable solution22:18
marunfungi: it's always risk vs reward22:18
anteayafungi: ++22:18
marunjeblair: we don't need a tag, though22:18
*** sabeen1 has quit IRC22:18
*** amitgandhinz has quit IRC22:18
marunjeblair: We can always just pin to something arbitrary22:18
marunjeblair: if that's a concern, maybe that's best.22:19
*** annegentle has quit IRC22:19
marunjeblair: we're already running into limitations around our current practice of global deps, no pinning22:19
*** dboik_ has quit IRC22:19
*** YorikSar has joined #openstack-infra22:19
marunjeblair: I get the motivation, but one-size-fits-all breaks down at the scale we're seeing.22:20
jeblairmarun: i don't think having projects say they need to use another project at this particular random git sha is a solution either.  that's well below the level of engineering this project is capable of.22:20
*** asselin has quit IRC22:20
jeblairmarun: no, this is not a breakdown of anything except thet fact that a proper inter-project api was not defined22:20
marunjeblair: I think disallowing a viable short-term approach on principle is self-defeating.22:20
jeblairbut used regardless22:20
marunjeblair: we need to do what we need to do22:20
*** gokrokve has quit IRC22:21
marunjeblair: so long as the end-goal is quality software that evolves to meet user demand, what, really, should be off the table?22:21
jeblairmarun: establishing an API when two modules depend on each other is pretty fundamental software engineering.  it needs to be no more complicated than that.22:21
marunjeblair: it takes time22:21
marunjeblair: and the pain during that time is real22:21
marunjeblair: again, why deny usage of a short-term fix to make that pain more bearable?22:22
marunjeblair: nobody is saying we want to do the wrong thing forever.22:22
fungithe term "throwing good money after bad" comes to mind. development effort spent implementing something you already have plans to rewrite22:23
marunjeblair: honestly, why is this something that infra needs or wants to be involved in?  The projects are responsible for converging on sanity every milestone.22:23
marunfungi: are you working on the project you're trying to dictate terms to?22:23
fungimarun: i don't personally have any inclination to work on that. but by the same token you shouldn't ask for my help implementing it22:24
marunfungi: I'm not.22:24
* fungi is speaking figuratively, on the assumption you're doing the same22:24
jeblairi think this would be a good time to end this conversation.22:24
*** tsg has quit IRC22:24
marunfungi: I'm asking to not be hamstrung by seemingly arbitrary decisions based more on principle than pragmatism.22:24
dtroyermarun: you are more than welcome to help define and build those interfaces.  I have less time that you do available for this, if you can't wait, then run along ahead and don't say we're using quality practices here.  when we talked about using devstack bits as a library, I made the apparently bad assumption that we would actually define the interface and not ust use whatever could be found22:24
mordredmarun: I do not think you're being hamstrung by anything22:25
mordredmarun: you came in here asking fora  human override on a system22:25
marunmordred: I think we'll have to disagree, then.22:25
mordredtaht is a cost22:25
mordredso we're going to ask to have that cost be justified22:25
mordredyou did not end up doing that22:25
mordredyou are still free to do whatever you feel like in neutron22:25
mordredbut if you want us to pefrom work, then you're going to have to convince us that the work is a good idea22:26
marunmordred: Are you suggesting that the functional testing we're doing doesn't have value, then?22:26
mordredno22:26
mordredI am not22:26
mordredI am suggesting that us bumping a change in the queue is the work you asked for22:26
marunmordred: Then why do you think that the cost here isn't justified?22:26
mordredoutside of that, I believe we're offering suggestions that we think that doing a pin of something in a neutorn repo is a bad idea - but none of us can stop you or are interested in doing so22:27
*** YorikSar has quit IRC22:27
marundtroyer: I'm not against defining an API.  It doesn't sound like helping me do so is a priority for you right now, though.22:27
mordredhowever, if it does break and causes more carnage and you need to come and ask for another bump of a change through a gate queue22:27
mordredyou're probably not going to get much sympathy22:27
marunmordred: I find it troubling that I need sympathy to do my job.22:28
mordredyou will need sympathy if you want to convince us to bypass the system that runs the automated testing for you22:28
mordredthere is a reason we don't do that lightly22:28
mordredbut, luckily, the system keeps chugging22:29
mordredand your change will land when it lands, like all changes do, and you will be happy, and the world will be good22:29
*** tsg has joined #openstack-infra22:31
*** YorikSar has joined #openstack-infra22:33
mtreinishclarkb, jeblair, fungi, mordred: would you be ok with me stealing subunit2html from the slave scripts and packaging that in os-testr?22:33
anteayain case this was missed, my first draft of an email to the infra list about discussion election tooling: https://etherpad.openstack.org/p/XU4qRouXIH22:34
openstackgerritClark Boylan proposed openstack-infra/nodepool: Support multiple image formats in a diskimage  https://review.openstack.org/12674722:34
*** pc_m has quit IRC22:34
anteayadiscussing22:34
clarkbmtreinish: yes please22:34
mordredmtreinish: fine by me22:34
jeblairmtreinish: i'm pretty sure the answer is yes.22:34
clarkbmordred: ^ 126747 should now be ready for rebasing and stuff22:34
fungimtreinish: seems fine, though we should figure out how to stop using the copy in the slave scripts22:34
mordredclarkb: woot!22:34
clarkbmordred: not sure what order you want that to go in with shade image uploads22:35
jeblairmtreinish, fungi: will we need to install os-testr on nodes like we do zuul?22:35
mordredclarkb: I'll poke at it later tonight and try to add it into the stack22:35
clarkbmordred: I even added a test though in my haste did not run it locally >_>22:35
mtreinishcool, then I'll start doing that22:35
jeblairmtreinish, fungi: or will it be a tox dep?22:35
mordredjeblair: I _think_ it goes into teh venvs with tox, yeah?22:35
mtreinishjeblair: not yet especially since I haven't pushed a release to pypi yet :)22:35
*** claudiub has quit IRC22:35
mtreinishI was waiting for it to be in gerrit before I did that22:35
mordredmtreinish: but the theory is that it will go into tox/test-requirements.txt yeah?22:35
fungibecomes a tox dep preferably? because preinstalling things in our images is something we should strive to avoid where possible22:36
mtreinishmordred: yeah I was going to add it as the test requirement everywhere instead of testrepository, etc.22:36
jeblairmtreinish, fungi, mordred: yeah, i think a tox dep is preferable22:36
mtreinishkinda like how we do it with hacking for flake8 and friends22:36
jeblairmtreinish: so far so good22:36
clarkbanteaya: just one question on the etherpad, otherwise it looks pretty good22:36
fungie.g. the more things we can fix live without needing to rebuild worker images, the better22:37
jeblairmtreinish: then what runs it -- will you add it to the commands in tox.ini so it runs after testr?22:37
jeblairmtreinish, fungi: so then the slave script does not need to run it separately?  or is that not possible because testr might exit with error and therefore not run it?22:37
anteayaclarkb: thank you22:38
mtreinishjeblair: well at least to start, it'll have to be installed globally in dsvm nodes because it's only used in d-g functions right now22:38
jeblairmtreinish: isn't it used in unit tests?22:38
mtreinishjeblair: oh yeah there too22:38
mtreinishjeblair: but ideally I'll eventually add a --html flag to os-testr to just dump a testr_results.html in the cwd22:40
jeblairmtreinish, fungi, mordred: this is probably where it gets tricky -- should this be in tox.ini in every project, or in jjb macros?22:40
jeblairmtreinish: oh you're replacing the testr invocation with an os-testr invocation?22:40
clarkboh that I would not do22:40
mtreinishjeblair: that's the eventual plan. It started as a replacement of my pretty_tox.sh tumors that were being spread to all the projects22:41
clarkbwe alrady have enough confusion using the test entrypoint from pbr22:41
fungia wrapper, huh?22:41
mtreinishclarkb: this would not use that at all22:41
*** annegentle has joined #openstack-infra22:41
clarkbmtreinish: yes I know, I am saying from experience with the pbr entrpoint using another wrapper to run testr is sadness22:41
clarkbbut maybe this ship has sailed22:41
clarkbI guess thats what pretty_tox.sh was (that name trips me up every time)22:42
mtreinishclarkb: I think it has, because right now we have a bash wrapper to get output people like, calling the pbr wrapper to default things in parallel, calling testr which is itself a wrapper22:42
clarkbmtreinish: yes, I got tricked by the name again22:42
*** smccully has quit IRC22:43
*** dboik_ has joined #openstack-infra22:43
*** eharney has quit IRC22:43
clarkbmordred: oh! I think I need to add image-types to the nodepool config validator stuff22:44
clarkbmordred: so my change isn't quite ready, working on that now22:44
*** YorikSar has quit IRC22:44
*** harlowja has quit IRC22:45
*** harlowja_ has joined #openstack-infra22:45
jeblairmtreinish: anyway, please do steal it, but let's keep talking about the best way to use it.  in general, having tox handle the dependency is good.  if it needs to be more than one step, we may want to do those in jjb macros for easy applicability across all the projects.  if you end up doing it in one step, maybe that's not an issue.22:45
*** peristeri has quit IRC22:45
jeblairmtreinish: also, consider whether the PTI should be updated for this (i don't know the answer right now)22:46
*** YorikSar has joined #openstack-infra22:46
jeblairmtreinish: http://governance.openstack.org/reference/project-testing-interface.html22:46
*** Sukhdev has quit IRC22:46
openstackgerritClark Boylan proposed openstack-infra/nodepool: Support multiple image formats in a diskimage  https://review.openstack.org/12674722:46
anteayathe syrup got away on me, it was further along than I thought :(22:48
mtreinishjeblair: yeah I was trying to do this within all the constraints listed there. If we want to we can try and force using os-testr as the testr interface in that, but I'm not sure that's needed22:48
*** rlucio has quit IRC22:52
ianwfungi: so https://github.com/pyca/pyopenssl/issues/202 is about the final wrap-up of the immediate centos7 memory issues I think.  ubuntu is just "lucky" it install python-openssl in its base image, which is why it hasn't ever noticed this in ci22:52
*** YorikSar has quit IRC22:53
clarkbianw: thanks for digging into that. Any ideas on how we can deal with it in the mean time?22:55
fungiianw: makes sense. so we're effectively testing much newer pyopenssl on centos7 than on trusty?22:55
*** YorikSar has joined #openstack-infra22:55
*** rlandy has quit IRC22:55
*** annegentle has quit IRC22:55
ianwclarkb: for centos, we can pre-install the packaged 0.13 version -> https://github.com/pyca/pyopenssl/issues/20222:55
fungii mean, we _want_ to not be installing the distro-packaged version on ubuntu as well... i expect swift is going to asplode all teh memories there too if we do that?22:55
ianwbut now i'm actually thinking that  pin in global-requirements is better22:56
ianw<0.14 ... which is the re-write to use cryptography rather than direct C bindings22:56
*** esker has quit IRC22:57
ianw^ i mean https://review.openstack.org/#/c/168217/22:57
*** markvoelker has quit IRC22:57
*** dimtruck is now known as zz_dimtruck22:58
*** gordc has quit IRC22:58
clarkbmight also be good to try a throwaway devstack change that installs latest pyOpenSSL everywhere to check that ubuntu falls over too?22:59
ianwclarkb: i can fire one off ... Ubuntu seems to have a little more headroom than centos22:59
ianwclarkb: but if you can log in once it's going, we can easily check relative usage v another vm23:00
clarkbI should be able to do that for the next hour or so, I have to afk after that though23:00
*** unicell has quit IRC23:02
*** unicell has joined #openstack-infra23:03
jeblairianw: also, are you aware of the dstat log saved on devstack runs?23:03
mtreinishanteaya: if you get a sec https://review.openstack.org/169042 should be pretty quick23:05
ianwjeblair: yes, but it's not really enough detail to debug issues like this. https://review.openstack.org/#/c/160232/ is something, it needs a few tweaks23:05
mtreinishclarkb: ^^^ I think you reviewd the d-g patch which that supersedes (I still need to abandon that)23:05
clarkbmtreinish: why the switch?23:06
jeblairianw: cool, you're ahead of me then.  :)23:06
ianwclarkb: https://review.openstack.org/#/c/16911223:06
mtreinishclarkb: because sdague said so and -1'd the d-g change23:07
mtreinishit doesn't really bug me one way or the other23:07
*** patrickeast_ has joined #openstack-infra23:08
clarkbok, that was my first suggestion, but ifgured if we were going to test with that opiton upstream it was probably fine23:08
clarkbwe should probably figure out what needs a DEVSTACK_GATE flag and what doesn't then clean the leftovers up23:08
*** patrickeast has quit IRC23:08
*** patrickeast_ is now known as patrickeast23:08
*** MarkAtwood has quit IRC23:08
jeblairyeah, i think the rule should be "things that alter devstack-gate's behavior"23:09
mtreinishclarkb: fwiw, the adding the flag approach was the only way I knew how to do it. It's been a while since I had to add a job23:09
jeblairthat _might_ be a little more subtle than it seems when you take branches into account, but maybe not.23:09
jeblairmtreinish: it's recent23:09
mtreinishyeah, from when we added devstack plugins iirc23:10
clarkbjeblair: so anything that d-g needs to operate on rather than straight pass through? sounds good to me23:10
jeblairyeah23:10
jeblairi mean, that's a starting place at least.  :)23:11
jeblairwe'll see what that doesn't work for23:11
anteayamtreinish: done23:11
anteayaI took this batch too far, it is on it's way to sugar23:11
* anteaya goes out to gather while there is still some daylight23:12
mtreinishanteaya: cool thanks23:12
anteayamtreinish: welcome23:12
jeblairShrews: mordred got both of us with 16867123:13
Shrewsjeblair: hehe23:14
clarkbianw: what are the magical incantations I should run on the ubuntu nodes running tests for 169112?23:15
*** tsg has quit IRC23:15
ianwclarkb: if you can just pastebin "ps --sort -pmem -eo pid:10,pmem:6,cputime:10,nlwp:8,wchan:25,comm:25,args:100" once devstack has run23:15
clarkbianw: thanks will get that up when devstack is done running23:16
*** melwitt has quit IRC23:17
*** Somay has quit IRC23:18
openstackgerritMerged openstack-infra/project-config: Use localrc passthrough for tempest no admin job  https://review.openstack.org/16904223:18
openstackgerritMerged openstack-infra/project-config: Use localrc passthrough for tempest no admin job  https://review.openstack.org/16904223:18
jeblairoh dear that looks weird.23:18
*** stevemar has quit IRC23:18
clarkbwow23:19
mtreinishooh, a double merge :)23:19
mtreinishdo I get two stackalytic points!23:19
*** baoli has joined #openstack-infra23:20
jeblairthe bot claims to have received two gerrit events...23:22
*** tsg has joined #openstack-infra23:22
*** ivar-lazzaro has quit IRC23:22
*** ddieterly has quit IRC23:23
mtreinishjeblair: fwiw, I got 2 emails too23:23
jeblairand zuul also got 2 notifications23:23
*** Somay has joined #openstack-infra23:24
*** alexpilotti has quit IRC23:24
*** baoli_ has joined #openstack-infra23:25
*** bswartz has joined #openstack-infra23:25
*** alexpilotti has joined #openstack-infra23:26
*** alexpilotti has quit IRC23:26
*** SumitNaiksatam has quit IRC23:28
*** baoli has quit IRC23:28
clarkbnice23:28
*** mattoliverau has quit IRC23:29
*** wenlock has quit IRC23:29
*** mattoliverau has joined #openstack-infra23:29
*** ghostpl_ has quit IRC23:30
jeblairclarkb, mordred: i think i can guess the answer to this... but is there a way for nodepool or shade to know what image formats to create/upload?23:33
clarkbjeblair: https://review.openstack.org/#/c/126747/723:34
jeblairclarkb: well, that's the change i'm trying to review23:34
jeblairclarkb: so i probably didn't ask the question well23:34
clarkbjeblair: do you mean know what image format based on the remote cloud in use?23:34
jeblairclarkb: particularly, i'm looking at the fact that we specify what image types to build.  i think we shouldn't have to... yes that.23:35
clarkbjeblair: that is possibly something that can go in os-client-config23:35
clarkbthen shade/nodepool can be smart about it when talking to dib23:35
jeblairclarkb: well, i don't think that os-client-config should have that information either23:35
jeblairclarkb: the more stuff that we put in os-client-config, the more broken openstack is with private clouds23:35
clarkbthen no, openstack does not tell you this info anywhere23:35
*** carl_baldwin has quit IRC23:36
jeblairclarkb: (and the more that os-client-config becomes the first-level support desk for all openstack clouds everywhere)23:36
clarkbwe even have to tell dib to use qcow2 version 0.10 for compat with hpcloud's qemu-img23:36
jeblairwe have failed23:36
*** asettle has quit IRC23:37
fungiseems like something glance should at least expose (if it can't just convert on the fly)23:37
jeblairclarkb: what do you think about instead of specifying that on the diskimage dict, we specify it for the provider, and then take the union of image types in providers and use that for all dib calls?23:38
*** hodos|2 has quit IRC23:38
*** dims has quit IRC23:38
jeblairit feels like it's really an attribute of the provider, and adding it to multiple diskimage sections is repetitive and potentially error prone ("i forgot to add qcow 0.10 to this image, so oops")23:39
jeblairclarkb: i should say, it's already on the provider in your change, so i guess i'm only proposing removing it from diskimage23:39
clarkbon the flip side its an image attribute23:39
clarkbhrm my change is diskimage based I thought23:40
jeblairclarkb: it is in both places on your change23:40
jeblairclarkb: so we would need to specify it in two places and make sure the things matched up.  i'm saying let the computer do it for us.  :)23:40
*** SumitNaiksatam has joined #openstack-infra23:40
clarkbianw: http://paste.openstack.org/show/197633/ http://paste.openstack.org/show/197635/23:41
*** Somay has quit IRC23:42
clarkbjeblair: I see, ya I can do that23:42
clarkbjeblair: I do not know how to model that for the qemu-img option23:42
jeblairclarkb: cool, i'll leave a message with a suggestion about that too23:43
clarkbpossibly we do a union of those options between image in provider and diskimage23:43
clarkbsince it is possible that you would want a special flag there that is provider independent23:43
clarkbwhereas the image type is more closely related to what the provider supports in glance23:44
ianwclarkb: yeah, so see any swift-* line there; compare with line 135 of https://etherpad.openstack.org/p/oom-in-rax-centos7-CI-job ... it usually takes up around 0.2% of a ci host's memory, it's showing up as ~0.6%23:44
clarkbianw: fun stuff23:44
ianwclarkb: interesting to see if it ooms ... even with centos it was a sometimes thing23:45
clarkbya, it wasn't 100%23:45
ianwdigging into whatever the heck cryptography is doing ... how could parsing a 1.4mb C file into an in-memory tree ever go wrong :)23:46
*** ujuc has quit IRC23:46
*** otter768 has joined #openstack-infra23:46
*** hichihara has joined #openstack-infra23:50
*** ZZelle_ has quit IRC23:50
*** otter768 has quit IRC23:51
*** ghostpl_ has joined #openstack-infra23:53
*** markvoelker has joined #openstack-infra23:53
jeblairclarkb, mordred: i have written a bit on 126747 that you both should probably read.23:53
*** carl_baldwin has joined #openstack-infra23:54
*** shakamunyi has quit IRC23:55
* clarkb reads23:55
*** annegentle has joined #openstack-infra23:56
*** Somay has joined #openstack-infra23:57
*** dannywilson has quit IRC23:58
*** weshay has quit IRC23:58
clarkbjeblair: I think the second option is much better. We specifically do not want to have several hundred images anymore23:58
*** ghostpl_ has quit IRC23:58
clarkbjeblair: basically image with name "foo" should be built once and be the same everywhere23:58
jeblairclarkb: yes, except we have to have 2 formats of every image23:58
openstackgerritRamy Asselin proposed openstack-infra/puppet-openstackci: WIP: Attempt to test log server  https://review.openstack.org/16911723:58
clarkbjeblair: ya but thats just a format not a different image23:59
jeblairclarkb: that's all i'm suggesting; what did i say wrong?23:59
asselin_nibalizer, clarkb ^^ quick script to test log server. There's probably a better way, but there's a start.23:59
jeblairclarkb: it's only a 'problem' because of the tripleo cloud -- we would create devstack-tripleo images for hp and rax, and devstack-trusty for tripleo23:59
anteayajeblair: I updated the agenda for tomorrow's meeting23:59

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!