Thursday, 2013-12-26

*** rakhmerov has joined #openstack-infra00:15
*** sarob has joined #openstack-infra00:16
*** rakhmerov has quit IRC00:19
*** sarob has quit IRC00:21
*** sarob has joined #openstack-infra00:24
*** sarob has quit IRC00:29
*** CaptTofu has joined #openstack-infra00:37
*** sarob has joined #openstack-infra00:46
*** sarob has quit IRC00:50
*** boris-42 has quit IRC00:58
*** nosnos has joined #openstack-infra01:08
*** yamahata has joined #openstack-infra01:14
*** mgagne has joined #openstack-infra01:15
*** rakhmerov has joined #openstack-infra01:16
*** rakhmerov has quit IRC01:20
*** CaptTofu has quit IRC01:22
*** yamahata has quit IRC01:30
*** yaguang has joined #openstack-infra01:36
*** sarob has joined #openstack-infra01:46
*** zehicle has joined #openstack-infra01:47
*** zehicle_at_dell has quit IRC01:49
*** sarob has quit IRC01:51
*** harlowja_away is now known as harlowja02:00
*** mihgen has joined #openstack-infra02:02
*** rakhmerov has joined #openstack-infra02:17
*** ljjjustin has joined #openstack-infra02:17
*** rongze has joined #openstack-infra02:18
*** rakhmerov has quit IRC02:21
*** rakhmerov has joined #openstack-infra02:24
*** rakhmerov has quit IRC02:24
*** dripton__ has joined #openstack-infra02:25
*** alexpilotti_ has joined #openstack-infra02:26
*** yamahata_ has joined #openstack-infra02:26
*** nicedice_ has joined #openstack-infra02:27
*** mihgen has quit IRC02:28
*** michchap_ has joined #openstack-infra02:29
*** nosnos_ has joined #openstack-infra02:31
*** mgagne1 has joined #openstack-infra02:31
*** fboz has joined #openstack-infra02:32
*** fboz is now known as fbo02:32
*** mgagne has quit IRC02:33
*** nosnos has quit IRC02:33
*** alexpilotti has quit IRC02:33
*** nicedice has quit IRC02:33
*** dripton_ has quit IRC02:33
*** michchap has quit IRC02:34
*** akscram has quit IRC02:34
*** yamahata__ has quit IRC02:34
*** fbo_away has quit IRC02:34
*** agordeev has quit IRC02:34
*** ianw has quit IRC02:34
*** alexpilotti_ is now known as alexpilotti02:34
*** agordeev has joined #openstack-infra02:38
*** akscram has joined #openstack-infra02:38
*** ianw has joined #openstack-infra02:40
*** rakhmerov has joined #openstack-infra02:55
*** notel has joined #openstack-infra03:17
*** rakhmerov has quit IRC03:28
*** sarob has joined #openstack-infra03:46
*** sarob has quit IRC03:48
*** rakhmerov has joined #openstack-infra03:49
*** harlowja is now known as harlowja_away03:55
*** ryanpetrello has joined #openstack-infra04:01
*** mgagne1 is now known as mgagne04:05
*** harlowja_away has quit IRC04:09
*** notel has quit IRC04:21
*** zehicle has quit IRC04:22
*** zehicle_at_dell has joined #openstack-infra04:27
*** chandankumar has joined #openstack-infra04:30
*** rongze has quit IRC04:46
*** sarob has joined #openstack-infra04:46
*** SergeyLukjanov has joined #openstack-infra04:46
*** sarob has quit IRC04:51
*** guohliu has joined #openstack-infra04:53
*** zehicle has joined #openstack-infra05:09
*** zehicle_at_dell has quit IRC05:12
*** ryanpetrello has quit IRC05:32
*** yfried has joined #openstack-infra05:43
*** sarob has joined #openstack-infra05:46
*** sarob has quit IRC05:51
*** praneshp has joined #openstack-infra05:58
*** nicedice_ has quit IRC06:02
*** dguitarbite has joined #openstack-infra06:10
*** dguitarbite__ has joined #openstack-infra06:11
*** rongze has joined #openstack-infra06:21
*** ljjjustin has quit IRC06:22
*** ljjjustin has joined #openstack-infra06:23
*** ljjjustin has quit IRC06:29
*** ljjjustin has joined #openstack-infra06:30
*** SergeyLukjanov has quit IRC06:31
*** vogxn has joined #openstack-infra06:42
*** vogxn has quit IRC06:43
*** sarob has joined #openstack-infra06:46
*** guohliu has quit IRC06:50
*** sarob has quit IRC06:50
*** garyk has joined #openstack-infra06:58
*** praneshp_ has joined #openstack-infra07:16
*** praneshp has quit IRC07:18
*** praneshp_ is now known as praneshp07:18
*** praneshp has quit IRC07:45
*** sarob has joined #openstack-infra07:46
*** sarob has quit IRC07:51
*** SergeyLukjanov has joined #openstack-infra07:51
*** SergeyLukjanov has quit IRC07:51
*** SergeyLukjanov has joined #openstack-infra08:03
*** dpyzhov has joined #openstack-infra08:04
*** amotoki has quit IRC08:07
*** SergeyLukjanov has quit IRC08:21
*** SergeyLukjanov has joined #openstack-infra08:30
*** dpyzhov has quit IRC08:38
*** sarob has joined #openstack-infra08:46
*** sarob has quit IRC08:51
*** dguit238 has joined #openstack-infra08:55
*** SergeyLukjanov has quit IRC08:55
*** dizquierdo has joined #openstack-infra08:56
*** ruhe has joined #openstack-infra09:15
*** yamahata has joined #openstack-infra09:26
*** ljjjustin has quit IRC09:31
*** DinaBelova has joined #openstack-infra09:34
*** yamahata has quit IRC09:35
*** dpyzhov has joined #openstack-infra09:36
*** boris-42 has joined #openstack-infra09:37
*** nosnos_ has quit IRC09:38
*** nosnos has joined #openstack-infra09:41
*** sarob has joined #openstack-infra09:46
*** sarob has quit IRC09:51
*** dguit238 has quit IRC09:52
*** dguitarbite has quit IRC09:52
*** dguitarbite__ has quit IRC09:52
*** ruhe is now known as ruhe_10:00
*** DinaBelova has quit IRC10:01
*** ruhe_ has quit IRC10:01
*** vkozhukalov has joined #openstack-infra10:02
*** SergeyLukjanov has joined #openstack-infra10:05
*** ruhe has joined #openstack-infra10:08
*** vkozhukalov_ has joined #openstack-infra10:09
*** ruhe has quit IRC10:13
*** dguitarbite_ has joined #openstack-infra10:14
*** vkozhukalov_ has quit IRC10:15
*** vkozhukalov has quit IRC10:15
*** vkozhukalov has joined #openstack-infra10:16
*** dguitarbite_ has quit IRC10:22
*** ruhe has joined #openstack-infra10:25
*** ruhe has quit IRC10:25
*** nosnos has quit IRC10:27
*** vkozhukalov has quit IRC10:35
*** chandankumar is now known as ciypro10:35
*** vkozhukalov has joined #openstack-infra10:38
*** yamahata has joined #openstack-infra10:41
*** gsamfira has quit IRC10:43
*** rongze has quit IRC10:45
*** sarob has joined #openstack-infra10:46
*** sarob has quit IRC10:51
*** rakhmerov1 has joined #openstack-infra10:52
*** rakhmerov has quit IRC10:55
*** lcestari has joined #openstack-infra10:56
*** gsamfira has joined #openstack-infra10:58
*** ruhe has joined #openstack-infra11:02
*** yaguang has quit IRC11:04
*** dpyzhov has quit IRC11:18
*** che-arne has joined #openstack-infra11:29
*** SergeyLukjanov is now known as _SergeyLukjanov11:35
*** _SergeyLukjanov has quit IRC11:36
*** ruhe has quit IRC11:39
*** ruhe has joined #openstack-infra11:42
*** ArxCruz has joined #openstack-infra11:45
*** sarob has joined #openstack-infra11:46
*** sarob has quit IRC11:51
*** ruhe is now known as ruhe_11:55
*** rongze has joined #openstack-infra11:56
*** ruhe_ is now known as ruhe11:57
*** rongze has quit IRC12:01
*** rongze has joined #openstack-infra12:03
*** ruhe has quit IRC12:11
*** ruhe has joined #openstack-infra12:14
*** dizquierdo has quit IRC12:24
*** yassine has joined #openstack-infra12:25
*** dpyzhov has joined #openstack-infra12:28
*** SergeyLukjanov has joined #openstack-infra12:35
*** dpyzhov has quit IRC12:36
*** plomakin has quit IRC12:38
*** che-arne has quit IRC12:41
*** sarob has joined #openstack-infra12:46
*** plomakin has joined #openstack-infra12:50
*** sarob has quit IRC12:51
*** tma996 has joined #openstack-infra12:53
*** ruhe has quit IRC13:04
*** dpyzhov has joined #openstack-infra13:11
*** ruhe has joined #openstack-infra13:17
*** DinaBelova has joined #openstack-infra13:21
*** vkozhukalov has quit IRC13:24
*** CaptTofu has joined #openstack-infra13:34
*** xchu has joined #openstack-infra13:41
*** ruhe has quit IRC13:42
*** sarob has joined #openstack-infra13:46
*** sarob has quit IRC13:51
*** NikitaKonovalov has joined #openstack-infra13:56
*** yassine has quit IRC14:05
*** yassine has joined #openstack-infra14:05
*** boris-42 has quit IRC14:15
*** dguitarbite_ has joined #openstack-infra14:25
*** alexpilotti has quit IRC14:30
*** SergeyLukjanov has quit IRC14:31
*** Grishkin has quit IRC14:31
*** SergeyLukjanov has joined #openstack-infra14:31
*** ryanpetrello has joined #openstack-infra14:33
*** alexpilotti has joined #openstack-infra14:37
*** yfried has quit IRC14:39
*** ruhe has joined #openstack-infra14:41
*** CaptTofu has quit IRC14:42
*** alexpilotti has quit IRC14:42
*** CaptTofu has joined #openstack-infra14:42
*** dizquierdo has joined #openstack-infra14:43
*** sarob has joined #openstack-infra14:46
*** sarob has quit IRC14:51
*** xchu has quit IRC14:53
*** rongze has quit IRC14:54
*** roeyc has joined #openstack-infra14:55
*** rnirmal has joined #openstack-infra15:02
roeycHi, I need help with running requests against gerrit rest api on review.openstack.org using curl, can anyone assist?15:03
roeycpls15:03
*** dguitarbite_ has quit IRC15:11
openstackgerritRoman Prykhodchenko proposed a change to openstack-infra/config: Add devstack-gate jobs for Ironic  https://review.openstack.org/5391715:15
*** yassine_ has joined #openstack-infra15:25
*** yassine has quit IRC15:26
fungiroeyc: it won't work yet. review.openstack.org is running gerrit 2.4.4 and the rest api wasn't added until a later release (we're working on upgrading, no current estimate on when we'll be in a position to transition to it though since there are a lot of changes)15:26
roeycfungi: got it, thanks!15:29
*** roeyc has quit IRC15:31
*** ciypro has quit IRC15:40
*** salv-orlando has joined #openstack-infra15:42
*** yfried has joined #openstack-infra15:43
*** sarob has joined #openstack-infra15:46
*** ruhe is now known as ruhe_15:47
*** yfried has quit IRC15:48
*** ruhe_ is now known as ruhe15:49
*** yfried has joined #openstack-infra15:49
*** sarob has quit IRC15:51
*** ruhe has quit IRC15:55
*** NikitaKonovalov has quit IRC15:58
*** SergeyLukjanov has quit IRC15:58
openstackgerritJames E. Blair proposed a change to openstack-infra/devstack-gate: Fix workspace setup  https://review.openstack.org/6415816:01
jeblairfungi: good news; the fix is to remove more code.  :)  ^16:01
*** ruhe has joined #openstack-infra16:01
*** harlowja has joined #openstack-infra16:04
*** rongze has joined #openstack-infra16:05
fungijeblair: the *best* kind of fix!16:05
openstackgerritJames E. Blair proposed a change to openstack-infra/devstack-gate: Fix workspace setup  https://review.openstack.org/6415816:05
jeblairfungi: hang on, im making it easier to diff16:05
fungicurrently internetting from the passenger seat on a road trip via wireless modem, so i'll be a little slower reviewing than usual anyway16:06
*** ruhe has quit IRC16:08
*** salv-orlando has quit IRC16:09
*** salv-orlando has joined #openstack-infra16:09
*** rongze has quit IRC16:10
openstackgerritJames E. Blair proposed a change to openstack-infra/devstack-gate: Fix workspace setup  https://review.openstack.org/6415816:12
jeblairfungi: okay, ps3 == ps1; ps2 == the change we reverted;  so you can diff ps2..ps3 to see what i just fixed, or diff HEAD..ps3 to see the whole thing.16:12
*** dpyzhov has quit IRC16:16
*** jaypipes has joined #openstack-infra16:17
*** salv-orlando_ has joined #openstack-infra16:17
fungiahh, that helps--thanks... i was digging up the old patchset and diffing16:20
*** salv-orlando has quit IRC16:22
*** salv-orlando_ is now known as salv-orlando16:22
DinaBelovahello guys! sorry to bother you, but I've got something extremely strange with one of my changes now... https://review.openstack.org/#/c/63097/16:23
DinaBelovaI've added new patchset today16:23
DinaBelovaAnd it was marked as failed just at this time16:24
DinaBelova*This change was unable to be automatically merged with the current state of the repository. Please rebase your change and upload a new patchset*16:24
DinaBelovaThe thing is that project repo itself has not changed16:24
DinaBelovaand new patchset is different from old one only by commit message16:24
DinaBelovaany ideas?16:24
jeblairIndexError: No item found with id 'origin/HEAD'16:25
jeblairDinaBelova: that change has never worked; probably no change has ever worked...16:26
DinaBelova0_016:26
jeblairfungi: ^ is that the manifestation of a bad clone?16:26
jeblairin zuul16:26
jeblairDinaBelova: should be fixed now; i rechecked the change16:27
DinaBelovajeblair, thanks a lot!16:27
*** ^d has joined #openstack-infra16:27
*** ^d has joined #openstack-infra16:27
fungijeblair: possibly--i'd never dissected one on zuul, but clarkb moved a few aside16:28
jeblairfungi: ok.  i assume so, istr it had something to do with HEAD not existing.16:28
DinaBelovawill wait what will happen :)16:28
openstackgerritJames E. Blair proposed a change to openstack-infra/config: Use OVERRIDE_ZUUL_BRANCH in alternate-branch jobs  https://review.openstack.org/6416216:29
fungithat's what i was seeing on gerrit and the git servers, but i had gotten the impression the checkouts on zuul manifested a different behavior (showed all files being deleted in the local working copy or sometrhnig16:29
*** SergeyLukjanov has joined #openstack-infra16:30
* fungi is having trouble seeing the screeen--sun is shining right on my lap16:30
openstackgerritJames E. Blair proposed a change to openstack-infra/config: Use OVERRIDE_ZUUL_BRANCH in alternate-branch jobs  https://review.openstack.org/6416216:30
*** ruhe has joined #openstack-infra16:32
openstackgerritJames E. Blair proposed a change to openstack-infra/devstack-gate: Add ZUUL_PROJECT to periodic test  https://review.openstack.org/6416316:33
openstackgerritJames E. Blair proposed a change to openstack-infra/config: Stop unsetting ZUUL_PROJECT in periodic jobs  https://review.openstack.org/6416416:35
*** UtahDave has joined #openstack-infra16:35
DinaBelovajeblair, that repo was not working :D That's fresh one slitted by one of our guys :) Seems like something went wrong :D Thanks one more time for help :)16:42
jeblairDinaBelova: yeah, there's a bug in the stackforge creation scripts and that happens to newly-imported projects sometimes. sorry about that.16:42
DinaBelovajeblair, good to know :)16:43
*** sarob has joined #openstack-infra16:46
jeblairfungi, clarkb: i see a path to removing periodic-devstack-gate.yaml and collapsing those jobs into the normal ones.16:47
SergeyLukjanovjeblair, it sounds great :)16:49
*** tma996 has quit IRC16:49
jeblairi want to make a small change to zuul first16:50
*** yfried has left #openstack-infra16:50
*** salv-orlando has quit IRC16:50
*** praneshp has joined #openstack-infra16:50
*** salv-orlando has joined #openstack-infra16:51
*** sarob has quit IRC16:51
*** boris-42 has joined #openstack-infra16:55
SergeyLukjanovjeblair, I see your change that renames ZUUL_BRANCH to OVERRIDE_ZUUL_BRANCH and here is a patch to add d-g to tempest https://review.openstack.org/#/c/53917/ that's already have 20 patch sets :)16:56
SergeyLukjanovare you expecting that it should be rebased on yours one?16:56
SergeyLukjanovI'm asking it to make a more correct infra reviews in future16:57
*** garyk has quit IRC16:58
*** salv-orlando_ has joined #openstack-infra17:00
jeblairSergeyLukjanov: since that one is so old, i'll rebase mine on his17:01
jeblair(and since it looks ready to go in)17:01
SergeyLukjanovjeblair, ok, thanks for clarification17:01
SergeyLukjanovjeblair, and yep, I think it's completed atm17:01
*** salv-orlando has quit IRC17:03
*** salv-orlando_ is now known as salv-orlando17:03
openstackgerritJames E. Blair proposed a change to openstack-infra/config: Use OVERRIDE_ZUUL_BRANCH in alternate-branch jobs  https://review.openstack.org/6416217:03
openstackgerritJames E. Blair proposed a change to openstack-infra/config: Stop unsetting ZUUL_PROJECT in periodic jobs  https://review.openstack.org/6416417:03
*** alexpilotti has joined #openstack-infra17:06
ArxCruzALL, Does anyone saw this problem when running tempest? http://paste.openstack.org/show/57224/17:08
jeblairArxCruz: i'm not aware of it.  I know of bug 1253896 but this looks different17:09
uvirtbotLaunchpad bug 1253896 in neutron "Attempts to verify guests are running via SSH fails. SSH connection to guest does not work." [Critical,In progress] https://launchpad.net/bugs/125389617:09
*** nicedice has joined #openstack-infra17:12
ArxCruzjeblair: doesn't seems to be the same problem :/17:16
*** nicedice has quit IRC17:16
ArxCruzit's trying to connect, but it seems the machine didn't answer17:16
jeblairArxCruz: i agree.  you might try asking in #openstack-qa, but i bet not many people are around today.17:16
ArxCruziptables is off, and selinux too17:16
*** nicedice has joined #openstack-infra17:17
openstackgerritJames E. Blair proposed a change to openstack-infra/config: Use OVERRIDE_ZUUL_BRANCH in alternate-branch jobs  https://review.openstack.org/6416217:21
openstackgerritJames E. Blair proposed a change to openstack-infra/config: Stop unsetting ZUUL_PROJECT in periodic jobs  https://review.openstack.org/6416417:21
openstackgerritJames E. Blair proposed a change to openstack-infra/config: Use OVERRIDE_ZUUL_BRANCH in alternate-branch jobs  https://review.openstack.org/6416217:23
openstackgerritJames E. Blair proposed a change to openstack-infra/config: Stop unsetting ZUUL_PROJECT in periodic jobs  https://review.openstack.org/6416417:23
*** dpyzhov has joined #openstack-infra17:26
*** sarob has joined #openstack-infra17:27
*** UtahDave has quit IRC17:29
*** yamahata has quit IRC17:30
ArxCruzjeblair: do you remember any option to pass to localrc to disable floating ips in devstack ?17:31
ArxCruzI'm not familiar with devstack, if you know, it will save me some google time17:31
jeblairArxCruz: i don't know17:31
*** boris-42 has quit IRC17:32
*** UtahDave has joined #openstack-infra17:32
*** hogepodge has joined #openstack-infra17:35
fungiokay, back. we stopped for lunch by chimney rock, and it was a complete wireless dead zone17:37
jeblairfungi: if you have a minute to review https://review.openstack.org/#/c/64164/ and it's dependency chain (3 changes), that would be swell17:37
jeblairfungi: did you try climbing to the top of the rock?  good line of sight there.  :)17:37
fungii did not try. instead i took the opportunity to climb to the top of a steak sandwich. having a look at those changes now17:39
openstackgerritJames E. Blair proposed a change to openstack-infra/config: Zuul layout: start using templates  https://review.openstack.org/6385417:43
jeblair(rebase on the ironic change ^ )17:44
*** nati_ueno has joined #openstack-infra17:48
*** DinaBelova has quit IRC17:48
*** garyk has joined #openstack-infra17:51
openstackgerritJames E. Blair proposed a change to openstack-infra/config: Use Zuul to send email for periodic jobs  https://review.openstack.org/6417617:51
*** nicedice has quit IRC17:52
*** nicedice has joined #openstack-infra17:53
*** ruhe is now known as ruhe_17:54
*** harlowja is now known as harlowja_away17:55
clarkbgood morning17:56
jeblairclarkb: good morning!17:56
SergeyLukjanovclarkb, morning!17:58
jeblairclarkb: i have exciting changes up for review! we're fixing the devstack-gate checkouts, and i'm about to remove the periodic job definitions! :)17:58
clarkbnice, I am looking at the fix for d-g workspace setup now17:58
jeblairclarkb: cool, when you're done with that, this is the tip of the config change tree: https://review.openstack.org/#/c/64176/17:58
*** ruhe_ has quit IRC18:00
openstackgerritA change was merged to openstack-infra/devstack-gate: Fix workspace setup  https://review.openstack.org/6415818:01
jeblairclarkb: ^ it's still worth your review of course; we can either patch or quick-revert if you find something18:01
clarkbjeblair: looks like OVERRIDE_ZUUL_REF handles the ZUUL_REF case as well?18:02
jeblairclarkb: yes (because it will substitute ZUUL_BRANCH for ZUUL_BRANCH in the ref, so it doesn't change it)18:04
*** CaptTofu_ has joined #openstack-infra18:04
*** CaptTofu has quit IRC18:05
fungijeblair: i think the ironic change is missing the timeout vars which got added to existing d-g jobs18:07
jeblairfungi: good catch, thanks18:07
clarkbremoving ZUUL_PROJECT unset works because setup_project will fall checkout the tip of $overridebranch properly now?18:08
jeblairfungi: i'll fix that, but it'll take me a few minutes to do18:08
jeblairclarkb: well, it was really unset because of the sanity check in devstack-gate (which was basically "hey, we ought to at least check out the zuul ref of the project zuul wants us to test")18:09
jeblairclarkb: unsetting zuul_project effectively disabled that check for the periodic jobs18:10
jeblairclarkb: but i removed that (because doing it after the refactor is very complicated), and i'm hoping the tests keep us from building a system that would do that.18:10
*** AaronGr_Zzz is now known as AaronGr18:11
clarkbjeblair: I made it through the patchseries they lgtm, but I think you are rebasing them shortly so I will reapply my votesafter that18:24
fungijeblair: on 64164 i think maybe you missed doing the same for swift-functional.yaml18:24
jeblairfungi: ack, thx18:25
*** rakhmerov1 has quit IRC18:25
* fungi is still slowly digging his way up through that review stack18:27
*** ruhe has joined #openstack-infra18:31
*** ruhe has quit IRC18:32
openstackgerritSergey Lukjanov proposed a change to openstack-infra/config: Add DEVSTACK_GATE_TIMEOUT to devstack-dsvm-neutron  https://review.openstack.org/6418218:32
SergeyLukjanovI find a job with missed DEVSTACK_GATE_TIMEOUT ^^18:32
jeblairSergeyLukjanov: cool, thanks.  I don't think that will touch any of the changes I'm making, so I think we can leave it outside the patch series for now18:33
SergeyLukjanovjeblair, yup, I think so18:33
*** nicedice has quit IRC18:33
jeblairSergeyLukjanov: oh, actually, it's going to conflict with the one i haven't pushed yet;  why don't you rebase it on that after i push it.18:34
jeblairSergeyLukjanov: (i suggest that because it will be smaller that way)18:34
*** nicedice has joined #openstack-infra18:34
SergeyLukjanovjeblair, sure18:34
SergeyLukjanovjeblair, I'm setting WIP to avoid another reviews and accident merges ;)18:35
openstackgerritJames E. Blair proposed a change to openstack-infra/config: Remove most periodic devstack-gate jobs  https://review.openstack.org/6418418:40
openstackgerritJames E. Blair proposed a change to openstack-infra/config: Use Zuul to send email for periodic jobs  https://review.openstack.org/6417618:40
openstackgerritJames E. Blair proposed a change to openstack-infra/config: Use OVERRIDE_ZUUL_BRANCH in alternate-branch jobs  https://review.openstack.org/6416218:40
openstackgerritJames E. Blair proposed a change to openstack-infra/config: Stop unsetting ZUUL_PROJECT in periodic jobs  https://review.openstack.org/6416418:40
openstackgerritJames E. Blair proposed a change to openstack-infra/config: Zuul layout: start using templates  https://review.openstack.org/6385418:40
openstackgerritJames E. Blair proposed a change to openstack-infra/config: Add devstack-gate jobs for Ironic  https://review.openstack.org/5391718:40
jeblairSergeyLukjanov: can you rebase on https://review.openstack.org/64184 ?18:41
SergeyLukjanovjeblair, k18:41
*** dripton__ has quit IRC18:42
*** dripton__ has joined #openstack-infra18:43
SergeyLukjanovjeblair. what do you think about mergin Ironic d-g jobs and I'll add D_G_TIMEOUT for them too?18:46
SergeyLukjanovin separated CR18:46
SergeyLukjanovto not grow and rebase the whole CR chain18:46
jeblairSergeyLukjanov: i updated https://review.openstack.org/#/c/53917/21 to have a timeout, and it's the first patch in the series now; i think it has everything it needs now?18:47
SergeyLukjanovjeblair, oh, missed this18:48
clarkbjeblair: can we safely change the log locations?18:48
jeblairclarkb: https://jenkins02.openstack.org/job/periodic-tempest-dsvm-full-havana/14/parameters/?18:48
jeblairclarkb: that's what's being passed as LOG_PATH.  it's different, but should be fine18:48
jeblairclarkb: i think we prepped for this step a long time ago but maybe never quite finished it.18:48
clarkbya I remember discussions of sorting by date18:49
SergeyLukjanovjeblair, it looks good to me, thank you18:49
jeblairclarkb: (future change will remove those macros)18:49
jeblair(i'm still writing more changes)18:49
clarkbanteaya: thank you for your response to that thread18:52
clarkbworth noting that every jenkins failure comment has a link to https://wiki.openstack.org/wiki/GerritJenkinsGit#Test_Failures I should probably respond to the list too18:52
anteayaclarkb: welcome18:53
openstackgerritJames E. Blair proposed a change to openstack-infra/config: Remove devstack-gate-periodic-jobs.yaml  https://review.openstack.org/6418618:53
anteayawould increase the possiblity that someone might alter their behaviour in a good way18:53
jeblairclarkb: ^ your refactor made that relatively simple!  i'm excited.18:53
*** yassine_ has quit IRC18:54
clarkbcool, I didn't expect it to be the end state (this is a large refactor too :) ) but it appears to have done its job of getting the ball rolling18:55
*** krotscheck has joined #openstack-infra18:55
clarkbanteaya: waiting for my client to show your response so that I can properly respond...18:55
*** rakhmerov has joined #openstack-infra18:56
anteayaclarkb: understood18:56
anteayaI'm looking at my typo18:56
anteayaalways one, I always manage at least one18:56
openstackgerritSergey Lukjanov proposed a change to openstack-infra/config: Add DEVSTACK_GATE_TIMEOUT to devstack-dsvm-neutron  https://review.openstack.org/6418218:56
openstackgerritA change was merged to openstack-infra/config: Add devstack-gate jobs for Ironic  https://review.openstack.org/5391718:57
openstackgerritJames E. Blair proposed a change to openstack-infra/config: Switch periodic-swift-dsvm-functional to use the template  https://review.openstack.org/6418718:57
*** dizquierdo has quit IRC18:58
*** Sukhdev has joined #openstack-infra18:58
clarkbout of curiousity is anyone else able to load review.o.o with chrome on an android phone? it seems to not work so well and shows the loading screen that usually indicates a broken gerrit build18:59
* anteaya heads to her phone19:00
*** yassine has joined #openstack-infra19:00
SergeyLukjanovclarkb, I have a problem with extremely slow working review.o.o using my home internet :(19:00
SergeyLukjanovclarkb, but it works ok my phone ;) (iOS)19:00
clarkbGWT compiles different js for different browsers I wonder if it is failing on this very small subset19:01
openstackgerritMichael Krotscheck proposed a change to openstack-infra/storyboard: Added .gitignore for IntelliJ specific configuration files  https://review.openstack.org/6295619:02
*** salv-orlando has quit IRC19:02
openstackgerritJames E. Blair proposed a change to openstack-infra/config: Template {gate,periodic}-gitdm  https://review.openstack.org/6418819:02
SukhdevI would like to request a service account so that we can run third party tests and post results/voting19:02
anteayaandroid phone built in browser, review.o.o loads fine19:03
* anteaya looks for chrome19:03
SergeyLukjanovSukhdev, http://ci.openstack.org/third_party.html#requesting-a-service-account19:03
*** reed has joined #openstack-infra19:03
SergeyLukjanovSukhdev, this page contains info about setting up third-party testing and about how to request a service account19:04
openstackgerritJames E. Blair proposed a change to openstack-infra/config: Stop using console-log-periodic in some jobs  https://review.openstack.org/6418919:05
jog0anteaya: I liked your response to the what happens if tests fail thread19:05
anteayajog0: thanks19:07
anteayaworking towards that pattern in neutron19:07
*** nicedice has quit IRC19:08
*** nicedice has joined #openstack-infra19:08
anteayaclarkb: chrome in android wors for review.o.o for me19:08
anteayaworks19:08
openstackgerritJames E. Blair proposed a change to openstack-infra/config: Switch periodic-swift-dsvm-functional to use the template  https://review.openstack.org/6418719:09
openstackgerritJames E. Blair proposed a change to openstack-infra/config: Remove devstack-gate-periodic-jobs.yaml  https://review.openstack.org/6418619:09
openstackgerritJames E. Blair proposed a change to openstack-infra/config: Stop using console-log-periodic in some jobs  https://review.openstack.org/6418919:09
openstackgerritJames E. Blair proposed a change to openstack-infra/config: Template {gate,periodic}-gitdm  https://review.openstack.org/6418819:09
clarkbanteaya: thank you for checking, could be a local problem (maybe I cached bad stuff)19:09
anteayawelcome, I haven't signed in, just looked at the first page19:09
anteayalooks accurate to my eyes19:09
*** rakhmerov has quit IRC19:10
clarkbcleared caches and it seems to load19:10
clarkbso I blame my client19:10
anteayawfm19:11
jeblairoh wow the python-bitrot jobs have bitrotted.19:11
jeblairthey are looking for nose_results.html and don't use gerrit-git-prep.19:12
jeblair(i'm about to remove them too!)19:12
jeblairoh, but gerrit-git-prep doesn't have a facility for doing OVERRIDE_ZUUL_BRANCH19:13
openstackgerritJames E. Blair proposed a change to openstack-infra/config: Update python-bitrot-jobs; remove periodic macros  https://review.openstack.org/6419119:18
jeblairclarkb, fungi, SergeyLukjanov: ^ okay, i think that's all the refactoring for now.19:18
SergeyLukjanovjeblair, cool, great changes ;)19:19
SergeyLukjanovjeblair, looks like it'll be much easier for new projects19:19
SergeyLukjanovand for adding new jobs too19:19
openstackgerritlifeless proposed a change to openstack-infra/config: Make gantt unittests non-voting.  https://review.openstack.org/6419219:20
jeblairSergeyLukjanov: i hope so.  also, i removed 414 lines from the jjb configuration.  it's 93% of it's previous size.  :)19:20
SergeyLukjanovjeblair :)19:22
lifelessjeblair: clarkb: if you guys have a timeslice to check that Gantt proposal above, would really appreciate it :)19:22
jeblairlifeless: already +2d; i'll aprv if clarkb or fungi don't review in a few.19:23
anteaya<- afk, y'know company and such19:23
lifelessjeblair: wicked thanks!19:24
lifelesswe may need to do the same thing to the client, but the server is where all the meat is, so I don't want to pre-judge that :)19:24
*** ^d is now known as ^demon|away19:27
* koolhead17 looks around19:27
brianclinejeblair: are you the best person to talk to about donating dedicated hardware boxen for infra/testing needs?19:30
jeblairbriancline: lifeless is probably the best person to talk to about that...19:31
SergeyLukjanovlifeless, +1 from me too19:31
jeblairbriancline: he has a plan to manage hardware resources using tripleo; then those can be supplied to the project using the openstack api, as our other test resources are19:32
jeblairbriancline: if you have resources that are already behind an openstack api (eg, a public cloud), then i'd be the one to talk to19:33
openstackgerritA change was merged to openstack-infra/zuul: Use timeout fixture (30 seconds) for tests  https://review.openstack.org/6384819:34
brianclinejeblair: thanks -- I'll email lifeless offline about hardware. as for any resources behind an openstack api, are you looking for those to be hardware?19:36
jeblairbriancline: we could use real and virtual machines; everything we run now is virtual; we expect the tripleo cloud to be able to provide both19:39
lifelessbriancline: hi19:40
*** UtahDave has quit IRC19:40
lifelesshttps://wiki.openstack.org/wiki/TripleO/TripleOCloud/Regions#Contributing_a_region_to_the_TripleOCloud19:40
lifelessbriancline: ^19:40
*** dstanek has joined #openstack-infra19:41
lifelessnote that the sizing we talk about is the minimum viable to be worth us maintaining; its not 'optimal' or 'desired size' :)19:41
jog0what is grenade forward19:42
jeblairjog0: proposed change on havana gets tested to make sure havana can still upgrade to master19:42
jog0jeblair: ahh thought so thanks19:43
clarkblifeless: approved19:43
jog0jeblair: so I am writting the infra code for https://review.openstack.org/#/q/status:open+project:openstack-dev/grenade+branch:master+topic:nova-compute,n,z19:43
jog0two questions: 1) is this a good env variable name https://review.openstack.org/#/c/64075/1/grenaderc19:44
jog02) what is a good name for the new job. the new job will only run on nova changes i think19:44
openstackgerritA change was merged to openstack-infra/config: Make gantt unittests non-voting.  https://review.openstack.org/6419219:46
jeblairjog0: regarding 1: i assume you'd need a variable for each service so that you could individually indicate to grenade whether it should expect them to rolling-upgrade?19:46
jeblairjog0: (assuming other services later grow the ability to rolling-upgrade)19:46
jeblairjog0: in which case, the var name makes sense to me.19:47
jeblairjog0: 2) i don't think the job needs to have nova in the name; so probably just 'gate-grenade-dsvm-rolling' and 'gate-grenade-dsvm-forward-rolling'19:48
clarkbjeblair: much of your JJB refactor stack is approved but the root of the tree isn't. Is it safe to approve the whole thing?19:48
jeblairjog0: and those can set the var as appropriate.19:48
jeblairclarkb: i believe os.19:49
jeblairso19:49
jog0jeblair: makes sense to me thanks19:49
jeblairjog0: the -rolling jobs should probably just indicate to devstack-gate that it should do rolling upgrades, and d-g should set the var in (1)19:50
*** yassine has quit IRC19:50
jog0jeblair: yup, writting it that way as we speak19:50
clarkbjeblair: approved19:50
jog0using the variable           export DEVSTACK_GATE_GRENADE_ROLLING=119:50
jeblairjog0: make sure you base your infra/config change on https://review.openstack.org/#/c/64191/19:51
jeblairjog0: we're merging a major refactor of d-g jobs right now19:51
jog0O_o19:51
openstackgerritA change was merged to openstack-infra/config: Use OVERRIDE_ZUUL_BRANCH in alternate-branch jobs  https://review.openstack.org/6416219:52
openstackgerritA change was merged to openstack-infra/config: Stop unsetting ZUUL_PROJECT in periodic jobs  https://review.openstack.org/6416419:52
jog0jeblair: which patch is the d-g file change?19:53
openstackgerritA change was merged to openstack-infra/config: Zuul layout: start using templates  https://review.openstack.org/6385419:53
jeblairjog0: just walk down the dependency tree from that change19:53
jeblairjog0: the short version of what might affect you is that there won't be any more separately defined periodic jobs (they use the normal devstack-gate job templates now)19:54
jog0ahh https://review.openstack.org/#/c/64186/2/modules/openstack_project/files/jenkins_job_builder/config/devstack-gate.yaml19:54
*** ryanpetrello has quit IRC19:55
*** AaronGr is now known as AaronGr_Zzz19:56
openstackgerritA change was merged to openstack-infra/config: Use Zuul to send email for periodic jobs  https://review.openstack.org/6417619:57
clarkbonce all of these changes go in I think we are ready for a mass purge of jenkins jobs19:58
clarkbI need to run an errand and grab some lunch but I can assist with that a bit later today19:58
*** dstanek has quit IRC19:59
openstackgerritA change was merged to openstack-infra/config: Remove most periodic devstack-gate jobs  https://review.openstack.org/6418420:00
openstackgerritA change was merged to openstack-infra/config: Add DEVSTACK_GATE_TIMEOUT to devstack-dsvm-neutron  https://review.openstack.org/6418220:00
openstackgerritA change was merged to openstack-infra/config: Remove devstack-gate-periodic-jobs.yaml  https://review.openstack.org/6418620:00
openstackgerritA change was merged to openstack-infra/config: Switch periodic-swift-dsvm-functional to use the template  https://review.openstack.org/6418720:00
openstackgerritA change was merged to openstack-infra/config: Template {gate,periodic}-gitdm  https://review.openstack.org/6418820:01
openstackgerritA change was merged to openstack-infra/config: Stop using console-log-periodic in some jobs  https://review.openstack.org/6418920:01
openstackgerritA change was merged to openstack-infra/config: Update python-bitrot-jobs; remove periodic macros  https://review.openstack.org/6419120:01
jeblairwhee.  a zuul merge bomb!  :)20:01
*** rakhmerov has joined #openstack-infra20:06
SergeyLukjanovyup :)20:06
*** nicedice has quit IRC20:08
*** nicedice has joined #openstack-infra20:08
fungiclarkb: i can do jenkins job cleanup later this evening if you want me to take care of it, i'm just hesitant to try it from the car (i'm similarly trying not to approve things which need babysitting at the moment since i never know when we're heading into another dead zone)20:09
*** reed has quit IRC20:09
clarkbfungi: I can ping when I am back from doing things20:10
*** rakhmerov has quit IRC20:10
openstackgerritJoe Gordon proposed a change to openstack-infra/devstack-gate: Add support for grenade-dsvm-rolling test  https://review.openstack.org/6419920:10
openstackgerritJoe Gordon proposed a change to openstack-infra/config: Add grenade-dsvm-rolling test  https://review.openstack.org/6420020:10
*** nicedice has quit IRC20:11
*** nicedice has joined #openstack-infra20:12
*** ryanpetrello has joined #openstack-infra20:17
*** lcestari has quit IRC20:20
*** Ryan_Lane has joined #openstack-infra20:22
openstackgerritJoe Gordon proposed a change to openstack-infra/devstack-gate: Add support for grenade-dsvm-rolling test  https://review.openstack.org/6419920:22
*** krotscheck has quit IRC20:34
*** banix has joined #openstack-infra20:42
banixHi; Would like to get a service account as described in http://ci.openstack.org/third_party.html. Anybody around to help?20:43
jeblairbanix: best bet is to send an email to openstack-infra@lists.o.o20:43
banixjeblair: Thanks. Will do that.20:44
*** yassine has joined #openstack-infra20:44
jeblairbanix: Sukhdev: we're working on clarifying/making explicit some expectations we have around 3rd party testing systems; it's a work in progress, but you might want to take a look at https://review.openstack.org/#/c/63478/3 to see what we need.20:48
*** ^demon|away is now known as ^d20:49
*** zehicle has quit IRC20:50
*** reed has joined #openstack-infra20:51
openstackgerritA change was merged to openstack-infra/config: Remove gate-cookbook-openstack-compute-chef-style from compute gating  https://review.openstack.org/6384320:52
banixjeblair: Thanks for the info.20:54
*** boris-42 has joined #openstack-infra20:54
*** dstanek has joined #openstack-infra21:02
SergeyLukjanovjeblair, I'm thinking about reporting version from zuul, but I'm not really sure will it be useful or not21:03
SergeyLukjanovif use pbr.version21:03
SergeyLukjanovjeblair, AFAIK zuul installed using pip install /bla/bla/bla21:04
*** sarob has quit IRC21:05
*** sarob has joined #openstack-infra21:05
*** krotscheck has joined #openstack-infra21:06
*** rakhmerov has joined #openstack-infra21:07
*** rakhmerov has quit IRC21:11
jeblairSergeyLukjanov: if it's installed with "pip install ." it should work...21:11
lifelessis zuul in shutdown mode or something? I did a recheck here - https://review.openstack.org/#/c/62692/ but it has no status after 1.5 hours21:11
SergeyLukjanovjeblair, I mean that it'll print something like 2.0.0.921:11
*** jcooley_ has joined #openstack-infra21:12
jeblairlifeless: it pops up a big orange banner in shutdown, so that shouldn't be it... will look in 1sec21:12
jeblairSergeyLukjanov: yeah, that's fine.  we'll know that's 9 commits past 2.0.0 on master.21:12
SergeyLukjanovjeblair, right, so, it's probably enough to understand revision21:13
jeblairSergeyLukjanov: yep.  the main thing is to be able to determine whether we're running a version of zuul that has added support for some new feature.21:14
*** gsamfira has quit IRC21:15
SergeyLukjanovjeblair, agreed, so, I'll use version_string()21:15
*** dstanek has quit IRC21:18
jeblairlifeless: ssh review gerrit query 62692 --submit-records21:18
jeblair  submitRecords:21:18
jeblair    status: RULE_ERROR21:18
jeblairlifeless: ^ that's a new thing from gerrit i've never seen.21:18
*** nicedice has quit IRC21:18
*** nicedice has joined #openstack-infra21:19
*** dims has quit IRC21:20
*** krotscheck has quit IRC21:20
lifelessjeblair: \o/21:20
lifelessjeblair: Do I win a prize?21:21
jeblairlifeless: SpamapS has a trophy on his desk; it's yours now. :)21:21
* lifeless dances the happy dance21:21
*** dstanek has joined #openstack-infra21:22
*** dims has joined #openstack-infra21:22
*** dpyzhov has quit IRC21:24
*** jcooley_ has quit IRC21:24
jeblairlifeless: there is a patchset 10 to that change that we (and zuul) can not see because it is a draft.21:27
jeblairdrafts are evil.21:27
*** jcooley_ has joined #openstack-infra21:28
*** dpyzhov has joined #openstack-infra21:29
*** jcooley_ has quit IRC21:33
*** dpyzhov has quit IRC21:35
*** gsamfira has joined #openstack-infra21:36
*** dpyzhov has joined #openstack-infra21:36
brianclinejeblair: lifeless: thanks for the info on test machines21:36
fungilifeless: that's bug 1260836 by the way21:37
uvirtbotLaunchpad bug 1260836 in zuul "Zuul can't enqueue a change whose child is a Gerrit draft" [Low,Confirmed] https://launchpad.net/bugs/126083621:37
*** dpyzhov has quit IRC21:37
clarkbfungi: I am just about back, not sure where you are21:38
openstackgerritSergey Lukjanov proposed a change to openstack-infra/zuul: Add zuul version to status.json  https://review.openstack.org/6420921:39
openstackgerritSergey Lukjanov proposed a change to openstack-infra/zuul: Collect and report last reconfigured timestamp  https://review.openstack.org/6384921:39
*** simonmcc has quit IRC21:40
*** simonmcc has joined #openstack-infra21:42
fungiclarkb: we're still a couple hours out. including another meal stop i don't anticipate being home until 00:00 utc. don't let me hold you up though21:43
clarkbok21:43
*** banix has quit IRC21:43
lifelessfungi: huh21:44
lifelessoh I missed the dep. DUH21:45
lifelessfungi: surely a recheck should be possible?21:45
jeblairlifeless, fungi: that bug doesn't describe exactly the situation lifeless encountered;  in this case it was the change itself that was a draft21:45
jeblairthere was no traceback in the zuul error log in this instance.21:46
jeblair(because gerrit told zuul it was not an actionable change)21:47
fungioh, i see21:47
lifelessI can't submit 62932 either; how does one take a change out of 'draft' ?21:50
fungiyes, so that is a somewhat more straightforward issue, but sometimes harder to spot because gerrit's webui doesn't do a great job of letting you know there's a newer patchset with a draft21:50
fungilifeless: you need the author of the draft change to un-draft (publish in gerrit terminology) that change, or get a gerrit admin to fiddle with the database21:51
*** sarob has quit IRC21:52
lifelessit doesn't say it's a draft in the webui at all21:52
lifelesshow does one tell that something is a draft?21:52
*** sarob has joined #openstack-infra21:52
fungiyeah, pretty horrible. the easiest way is to try to review it, and you won't get any code review voting options21:53
jeblairi think the lack of review columns is an indirect indicator as well21:53
*** loq_mac has joined #openstack-infra21:53
fungiahh, right, it does omit those21:53
jeblair(as is the lack of "needs [code review] [approved]", etc...21:53
jeblairit's just terrible.21:53
clarkbfungi: you mentioned JJB could list a diff of jobs it knows about and those in jenkins itself? I don't see that option21:54
lifeless62932 has review columns21:54
lifelessand I can review it21:54
clarkbfungi: or are you outputting a JJB list then running find in ~jenkins to get a list for jenkins?21:54
openstackgerritSergey Lukjanov proposed a change to openstack-infra/config: Display zuul version from zuul/status  https://review.openstack.org/6421121:54
openstackgerritSergey Lukjanov proposed a change to openstack-infra/config: Display last reconfigured time from zuul/status  https://review.openstack.org/6385021:54
jeblairlifeless: i don't think it's a draft21:55
lifelessjeblair: ok.21:55
Mithrandirclarkb: it can delete obsolete jobs21:55
lifelessSo - how do I get it rechecked ;)21:55
fungiclarkb: i used to do what you describe, but now i do what Mithrandir describes (since it grew that option)21:55
clarkbwhat is the option? I see delete and delete-all21:55
Mithrandirupdate --delete-obsolete, iirc?21:56
MithrandirI don't remember what I called it21:56
Mithrandiror --delete-old21:56
Mithrandir    parser_update.add_argument('--delete-old', help='Delete obsolete jobs',21:56
Mithrandirso yeah21:56
clarkbah its an option under update not a distinct command21:57
*** sarob has quit IRC21:57
Mithrandiryeah, you kinda need to give it a list of jobs that it should know about21:57
Mithrandirand I should add that "add a tag and only delete the ones with the right tag" sometime in my CFT21:57
clarkbif you don't give it a list does it use the list of jobs built from the config? looks like it might21:58
lifelesscould someone look for errors for 62932 ? recheck no bug -> nothing on the review or in status../zuul21:59
Mithrandirclarkb: yes, I run it like jjb update --delete-old $dir  and it just nukes those that are no longer present in $dir21:59
jeblairlifeless: yeah, i was just doing that22:00
lifelessjeblair: thanks!22:00
clarkbMithrandir: awesome thank you. I think I will just update our puppet to do that then22:00
Mithrandir:-)22:00
jeblairlifeless: 62932 is needed by 62692 which is needed by 63604 which has only one patchset and it's a draft22:01
jeblairlifeless: that case _is_ the bug fungi pointed out earlier22:01
lifelessoh22:01
lifelessso the thing that's *depended on* gets bust?22:02
openstackgerritSergey Lukjanov proposed a change to openstack-infra/zuul: Fix and enable H4XX (docstrings) and H902 ('not in')  https://review.openstack.org/6392322:02
openstackgerritSergey Lukjanov proposed a change to openstack-infra/zuul: Fix and enable H2XX (py3k compatibility issues)  https://review.openstack.org/6392222:02
openstackgerritSergey Lukjanov proposed a change to openstack-infra/zuul: Enable hacking, ignore only failing checks  https://review.openstack.org/6392122:02
openstackgerritClark Boylan proposed a change to openstack-infra/config: Use JJB --delete-old option to keep job list clean  https://review.openstack.org/6421222:02
clarkbfungi: jeblair Mithrandir ^22:02
jeblairlifeless: yes.  i do consider that a bug in zuul (inasmuch as zuul should be able to deal with gerrit's insanity without throwing exceptions)22:03
*** senk has joined #openstack-infra22:04
lifelesshuh, I thought this was filter-branched but it looks like it has the whole nova history. Shrug, I guess.22:04
fungiclarkb: yeah, i agree it's time. i did it by hand the first few times just so i had some control and could double-check it didn't go off the rails for some reason, but it's worked fine for me each time so should be safe22:04
Mithrandirthe only case I know where people get upset about the delete-old functionality is some people who use the description field as a notetaking place, and jjb will always write a description that way22:05
clarkbdkranz: are you around? I am thinking about your post to the infra list and want to make sure I understand one thing first22:05
jeblairlifeless: in the mean time publish or abandon 63604 is the only fix22:05
*** loquacities has joined #openstack-infra22:05
clarkbdkranz: is the problem that waiting for longer jobs makes it harder to iterate on the shorter ones?22:06
*** loq_mac has quit IRC22:07
*** loquacities is now known as loq_mac22:07
*** rakhmerov has joined #openstack-infra22:07
*** senk has quit IRC22:08
*** dpyzhov has joined #openstack-infra22:08
*** senk has joined #openstack-infra22:09
jog0in devstack-vm-gate-wrap do I need to sudo in orde rtogzip?22:10
jog0I assume not, but the only cases I see use sudo. But is that because those files are not created by devstack-vm-gate-wrap22:10
jeblairjog0: yes, they're copied or written into the directory with sudo, so you generally need to sudo gzip them.  at the end they are chowned.22:11
fungijog0: the wrap script is running as jenkins, while other files may be generated by stack or tempest accounts22:11
jog0thanks22:12
fungi(or root, possibly)22:12
*** Sukhdev has quit IRC22:12
*** rakhmerov has quit IRC22:12
openstackgerritJoe Gordon proposed a change to openstack-infra/devstack-gate: Collect list of installed packages at end of run  https://review.openstack.org/6355122:16
openstackgerritJames E. Blair proposed a change to openstack-infra/config: Use templates for doc publish jobs  https://review.openstack.org/6421422:17
openstackgerritJoe Gordon proposed a change to openstack-infra/devstack-gate: Add support for grenade-dsvm-rolling test  https://review.openstack.org/6419922:21
*** senk has quit IRC22:22
openstackgerritJoe Gordon proposed a change to openstack-infra/config: Add grenade-dsvm-rolling test  https://review.openstack.org/6420022:23
*** ^d has quit IRC22:24
clarkbjog0: quick question. Is there value in testing both non rolling upgrades and rolling upgrades? or can we just test the rolling upgrades as that should cover the non rolling upgrades as well22:29
clarkbinitially we may need two jobs in order to get rolling upgrades working, but once they are working can that job replace the existing job?22:29
jog0clarkb: actually I am not sure.22:29
*** salv-orlando has joined #openstack-infra22:30
*** SergeyLukjanov has quit IRC22:30
jog0the rolling upgrade test tests the during the upgrade state: old nova-compute plus new rest22:30
jog0that way we can run tempest against it22:30
jog0so it doesn't cover any upgrade issues in swaping out the old nova-compute with a new one (mainly config options)22:31
jog0so I don't think we can replace the existing job just yet22:31
*** dims has quit IRC22:31
clarkbhow si that a rolling upgrade?22:31
clarkbthat is half an upgrade22:31
jog0clarkb: with regard to 63551 http://stackoverflow.com/questions/592620/how-to-check-if-a-program-exists-from-a-bash-script22:33
jog0clarkb: yes its half the upgrade correct. rolling upgrade in this case means supporting the interim state22:33
jog0otherwise we would have to run tempest twice in one job22:35
jog0which would be too long22:35
clarkbjog0: the job isn't worth much otherwise imo22:35
clarkbthe end goal is to run new everything22:35
clarkbrunning old compute at an interim step is just one step in getting there22:36
jog0yes, but the hard part is making sure that everything works in the interim state when its a mix22:36
jog0clarkb: in nova we expect there to be many nova-computes and don't expect deployers to be want to upgrade them all all at ounce22:36
jog0once22:36
clarkbjog0: I understand that22:36
clarkbjog0: but you aren't testing a rolling upgrade...22:36
jog0so this would be the ideal rolling nova-compute test22:37
clarkbyou are testing old nova compute with new everything else22:37
jog0assume two nova-computes. 1) set up old, pull up all but nova-compute. run tempest. pull up one nova-compute, run tempest. pull up second nova-compute so now all new. run tempest again22:38
jog0clarkb: this is is a test to make sure rolling upgrade works not a test that does rolling upgrades22:38
fungijeblair: 64214 starts to make it obvious that we have some post-versioned projects updating docs between releases22:38
clarkbjog0: right that is the problem :)22:38
jog0clarkb: and the answer is tripleo22:38
clarkbjog0: grenade tests that upgrades work22:38
jog0but until then22:39
clarkbthis doesn't22:39
jog0clarkb: thats why we shouldn't replace the standard grenade test22:39
clarkbjog0: I get that now. But I also think the new proposed test is confusing and not doing what people would expect of it22:39
jog0also this test has already found at least 1 bug in nova22:39
clarkbit should either test a rolling upgrade or not be called grenade rolling22:40
jog0clarkb: perhaps the naming is a little confusing22:40
jog0what would you propse calling it22:40
clarkb{pipeline}-dvsm-tempest-old-nova-new-splat ? I dunno22:40
clarkbbut this isnt a rolling grenade upgrade test22:41
jog0this is a test to make sure rolling upgrades work. not a test that rolls grenade22:41
clarkbright but that isn't what you are testing...22:41
clarkbyou are testing an intermediate step (which is fine on its own) but isn't what you are claiming to test22:41
clarkbthe claim is that an upgrade of openstack is being performed in a rolling fashion and that upgrade is being tested22:42
jog0technically it is, we just don't finish the upgrade ;)22:42
jog0anyway I agree the name is confusing22:42
jog0but before changing it I want to see what others say22:42
openstackgerritA change was merged to openstack-infra/config: Use templates for doc publish jobs  https://review.openstack.org/6421422:43
clarkbpersonally I would prefer we test the end to end upgrade and keep the name22:43
jog0see feel free to -1 on the name.  I dom't like old-new-splat much either22:43
jog0how can we test end to end upgrade?22:43
jog0clarkb: without running tempest twice serially?22:43
clarkbexercises22:44
jog0that doesn't count22:45
jog0thats not enough of a test22:45
lifelessjeblair: thanks22:45
clarkbwhy do the tempest tests have to be run serially twice?22:45
fungiokay, almost home but pulling off for dinner first. dropping offline for a little while22:45
jog0we want to make sure that nova works with new everythign plus old compute22:45
jog0and new everything22:45
lifelessjog0: two test environments22:45
jog0'make sure works' == passes tempest22:45
lifelessjog0: one that tests new everything22:45
jog0lifeless: yup22:45
jog0thats what we do22:45
lifelessjog0: one that tests new + old22:46
lifelessjog0: where is the serially in that22:46
lifeless?22:46
jog0lifeless: yes and yes22:46
*** loquacities has joined #openstack-infra22:46
jog0lifeless: clarkb 'clarkb> personally I would prefer we test the end to end upgrade and keep the  name22:46
*** dims has joined #openstack-infra22:47
jog0lifeless: clarkb wants a end to end rolling upgrade test.22:47
clarkbtoday we test end to end upgrades with grenade. If someone tells me grenade passed I assume that one can go from one release to another or last release to master22:47
clarkbthat is what grenade does22:47
clarkbthe proposed test that calls itself a rolling upgrade test does not test that22:47
lifelessjog0: three environmetns22:47
lifelessjog0: :)22:47
*** senk has joined #openstack-infra22:48
lifelessjog0: do the upgrade under load22:48
jog0lifeless: baby steps22:48
*** loq_mac has quit IRC22:48
jog0but I really want to see that one22:48
clarkbjog0: so we need something after the first step and something after the second step (full upgrade). I don't see why both have to be serial tempest22:48
jog0lifeless: patch set in question https://review.openstack.org/#/q/status:open+branch:master+topic:nova-compute,n,z22:48
lifelessjog0: ok so skip the load bit, but rolling seems to be different to the specific mixed scenarios22:48
jog0lifeless: well rolling means rolling upgrades of nova-computes22:49
jog0we really want to test two nova-computes22:49
clarkbboth could be parallel tempest if we ignore grizzle to havana22:49
jog0but thats out of grenade/devstacks scope today22:49
jog0clarkb: so we have two tests, one after first step, and one after second step. thats what  I am proposing22:50
lifelessjog0: so rolling means live migrating things etc?22:50
jog0lifeless: rolling means roll out nova-computes less then all at once22:50
jog0this is a rpc versioning test22:50
jog0I am refering to the code up now, not what we want to do in the future22:51
clarkbjog0: I only see tests for the first step22:51
jog0clarkb: second step is original grenade22:51
clarkbno original grenade goes from old everything to new everything22:51
jog0right, and the new job goes from old everything to new everything *except* nova-compute22:51
clarkbright so you are missing old nova compute + new everything to new nova compute22:52
jog0ahh I see what your saying you want a test that does just that.22:52
jog0not sure how to do that easily22:52
clarkbor combine that in the proposed tests22:52
clarkbbecause you aren't doing a rolling upgrade to new everything :)22:52
jog0clarkb: https://review.openstack.org/#/c/64074/1/upgrade-nova22:53
jog0this is the crux of the patchset22:53
jog0rolling means support rolling upgrades of nova-computes22:53
jog0we want to support new + old nova-computes at once22:53
jog0rolling=don't require all of a specific service to be upgraded in lockstep.22:54
clarkbright22:54
jog0rolling!=live22:54
jog0or now downtime22:54
jog0no22:54
clarkbright22:54
clarkbbut it does mean go from old to new22:55
jog0not exactly22:55
clarkbnot go from old to old mixed with new and stop there22:55
jog0so lets take a step back22:55
jog0so we have three states: (A) old -> (B) new+old compute -> (C) new22:56
clarkbjog0: don't think of it in states, the state transitions are what is important here22:57
clarkbyou have 2 transitions22:57
jog0right22:57
jog0a->b, b->c22:57
jog0but the assumption is b->c is comparable to a->c22:57
jog0in issues and gotchas etc22:57
*** dstanek has quit IRC22:58
jog0because grenade doesn't do live upgrades,  they are very similar22:59
jog0because there is no load during a transition22:59
*** jerryz has joined #openstack-infra23:00
jog0so state b is actually many substates23:00
clarkbI don't think that is a fair assumption because the system is running after a->b and can change state within itself23:00
jog0given n computes we want to support upgrading them one by one. so b has n states23:00
clarkbit is possible that it works fine after a->c and a->b but not b->c23:02
jog0clarkb: thats possible but for now, I am assuming the risk of somthing breaking after running a->b is minimal23:02
jog0clarkb: can you give a hypothetical example23:03
openstackgerritJames E. Blair proposed a change to openstack-infra/config: Sort projects in layout.yaml  https://review.openstack.org/6421623:03
lifelessrolling is a horrid name :)23:03
jeblairmy eyes are bleeding after that ^23:03
lifelessgotta run, ciao23:03
clarkbjog0: mostly thinking of DB state. old compute + new nova may change DB in ways that work in that state but don't once you have new everything23:04
jeblairlifeless: i made all _kinds_ of assumptions based on the word 'rolling', which i suspect the last 200 lines of scrollback invalidate.23:04
lifelessjeblair: yes.23:05
*** jog0 has quit IRC23:05
*** vipul has quit IRC23:05
*** jog0 has joined #openstack-infra23:05
*** GheRivero has quit IRC23:05
*** dstufft has quit IRC23:05
* lifeless runs23:06
*** dstufft has joined #openstack-infra23:06
*** GheRivero has joined #openstack-infra23:06
*** vipul has joined #openstack-infra23:06
*** loquacities has quit IRC23:06
jog0clarkb: I need to head out for a bit23:07
jog0but in short I agree the name isn't good, put some of your thoughts on that in the reviews23:08
*** rakhmerov has joined #openstack-infra23:09
jog0as for the risk from b->c I agree its non-zero but not even sure of a hypothetical issue. Also this test is not the end all for upgrade testing, its very much short term solution until we can use tripleo23:09
jog0one option is to add a step onto this grenade test that upgrades nova-compute and then runs the smoketests23:11
jog0and run that along the regular grenade test23:12
clarkbjog0: I have commented on the config change23:12
*** gsamfira has quit IRC23:12
jog0but I don't think that is enough to properlly test b->c if that is what we are concerned about23:12
jog0clarkb: thanks23:12
* jog0 takes lifeless's queue and goes for an actual run23:12
*** rakhmerov has quit IRC23:13
*** loq_mac has joined #openstack-infra23:15
clarkbjeblair: can I get your thoughts on https://review.openstack.org/#/c/64212/1 ?23:17
*** krotscheck has joined #openstack-infra23:17
*** alexpilotti has quit IRC23:19
*** zehicle_at_dell has joined #openstack-infra23:21
*** nicedice has quit IRC23:22
dkranzclarkb: Sorry, at dinner.23:22
*** nicedice has joined #openstack-infra23:22
dkranzclarkb: Yes, to your question.23:22
dkranzclarkb: Usually a check experimental by a patch author is to run a particular job, or a few related jobs.23:23
dkranzclarkb: Since the others are non-voting there is not reason to run them23:23
clarkbdkranz: right we can optimize the number of jobs run but this is infrequent enough that I wonder if the effort is worth it23:24
clarkbso trying to understand the problem you want to solve23:24
clarkbfor iterating you can view results via jenkins and zuul and pushing a new patchset allows you to kill running jobs23:25
*** yassine has quit IRC23:25
dkranzclarkb: I don't know, since I don't know how much effort it is.23:25
dkranzclarkb: If there are other ways to avoid waiting for the other jobs that would be fine too perhaps.23:25
*** nicedice has quit IRC23:25
clarkband idealy jobs are experimental for a couple weeks then move to silent/check/gate23:25
*** nicedice has joined #openstack-infra23:26
clarkbdkranz: I think it would need patching zuul23:26
dkranzclarkb: It is really all the new projects that are the issue: ironic, savanna, etc.23:26
dkranzclarkb: with more coming23:26
clarkbbut what is the issue23:26
dkranzclarkb: They will be around for awhile23:27
clarkbrunning a lot of jobs isnt an issue23:27
dkranzclarkb: I know23:27
dkranzclarkb: I was thinking about resubmit and test turnaround23:27
dkranzclarkb: and it was just a suggestion23:27
clarkbI know but I can't respond to it without knowing the problem it intends to solve23:28
clarkbso iteration is the issue?23:28
dkranzclarkb: I thought I said it was jobs taking a long time to run, but you said there were other ways to avoid that.23:29
*** jhesketh has joined #openstack-infra23:30
clarkbwell you said waste time running many jobs the invoker does not care about which shouldnt be a problem23:30
dkranzclarkb: Yes, iteration.23:30
dkranzclarkb: I meant waste time waiting for them to finish.23:31
clarkbI see23:31
dkranzclarkb: But if you don't think it is as big deal, and no one else does, then so be it.23:31
krotscheckclarkb- jeblair recommended yesterday that I just create a workspace-local install of nodejs to support my build tools. Any objection to that? It might mean compiling from source every build (trying to avoid that).23:33
krotscheckMy current working build has it creating a workspace/.local/bin that's then added to the PATH.23:34
clarkbkrotscheck: that should work. ccache is installed on the slaves so compilation time shouldn't be terrible23:35
clarkbdkranz: I think it is worth talking about, I will respond to the list now23:35
krotscheckclarkb: Perfect, that gives me the flexibility I need :)23:36
dkranzclarkb: ok, thanks23:36
dkranzclarkb: One more thing is that we don't want to put a 'time quota' on experimental jobs I think and if they always get run then we will need to do that.23:37
dkranzclarkb: This problem affects tempest more than the individual projects.23:37
clarkbdkranz: why would we need a time quota?23:39
dkranzclarkb: If I have an experimental job for project foo I might want it to take a long time for my purposes.23:39
dkranzclarkb: But if I stick it on the tempest experimental queue, then all users of that feature in other projects have to wait. Unless I am missing something.23:40
clarkbdkranz: you don't have to wait... zuul and jenkins give real time data23:40
dkranzclarkb: ok, I see23:40
dkranzclarkb: You just have to wait to get the email that it finished23:41
dkranzclarkb: Got to run now. I'll look at your email.23:43
jeblairfungi: recommend you revisit your +2 of https://review.openstack.org/#/c/63934/ (see my comments)23:43
clarkbdkranz: ok, just about done with it23:44
*** gsamfira has joined #openstack-infra23:44
*** gsamfira has quit IRC23:46
openstackgerritA change was merged to openstack-infra/gear: Add statsd support  https://review.openstack.org/6318323:51
*** rnirmal has quit IRC23:52
jeblairtagged gear 0.5.023:53
*** dstanek has joined #openstack-infra23:54
*** dstanek has quit IRC23:59

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!