Friday, 2017-02-17

clarkbtesting now00:00
mordredmyslqdump is running00:00
openstackgerritMonty Taylor proposed openstack-infra/shade master: DNM - Patch to verify 2.0 usage payload behavior  https://review.openstack.org/43448200:00
mordredWOOT!00:00
clarkbleaving comments worked too00:00
mordreddingdong the witch is dead00:00
JayFWait, WHAT00:00
*** wolverineav has joined #openstack-infra00:00
JayFis gerrit working during backups?!00:00
*** sflanigan has joined #openstack-infra00:00
*** rbuzatu has joined #openstack-infra00:01
fungishocked and awed00:01
mordred#success Gerrit pause at 00:00UTC has been erradicated - gerrit works during backups00:01
JayFholy crap00:01
mordredisn't that how that's supposed to work?00:01
JayFthe timing can't be a coincidence00:01
clarkbwe just needed someone that knows how mysql works to spend 5 minutes thinking about this problem00:01
JayFyou all are gunning for free beers at the ptg00:01
JayFlol00:01
JayFjlvillal: ^^^ !!!!00:01
openstackstatusmordred: Added success to Success page00:01
dasmfree beers? during ptg?00:01
dasmfor everyone?!00:01
mordredJayF: I can fix all of the mysql problems - I just have to get pissed off enough first00:01
dasmor just infra folks? :)00:01
JayFfor whoever fixed this, one from me, lol00:01
JayFI usually stop around 0000 UTC, so I've had more than once where I went to submit a review and ended up being hung up for minutes00:02
clarkbetherpad should also similarly be happy now00:02
fungiif by beer you mean using gerrit, and if by ptg you mean between 00:00 and 00:05 utc...00:02
jlvillalJayF, Sweet :)00:02
mordredfungi: is that NOT what everyone means by beer and ptg?00:02
dasmJayF: doesn't mean it: it's time to call it a day? :)00:03
fungiyeah, pro'lly00:03
clarkb#success fixed infra db backups so everything doesn't pause at 0000UTC00:03
JayFthat's seriously awesome, ty for the fix00:03
openstackstatusclarkb: Added success to Success page00:03
clarkbmordred: see I was always under the impression that the reason xtrabackup existed was to solve this problem. So extra happy that it can be solved with simpler tools00:04
*** erlon has quit IRC00:05
*** kzaitsev_mb has joined #openstack-infra00:05
mordredclarkb: xtrabackup is more efficient than mysqldump - it does a live binary copy of the data, rather than translating out into string sql statements00:05
*** rbuzatu has quit IRC00:05
*** wolverineav has quit IRC00:05
mordredso for some data, there's just no way mysqldump will ever work00:05
JayFis 'xtrabackup' what innodump is called nowadays?00:06
JayFI know I used to do live online binary backups of innodb mysql dbs using some tool wehad to pay for, that sounds suspiciously similar to that00:06
mordredJayF: yah - it's a fork of that done by percona along with their xtradb work00:06
JayFI should like, read up on modern mysql stuff00:06
JayFIt's gotta be way better than when I had to run it all the time :)00:06
mordredbut, as with many things mysql, it's not actually necessary - it's just most people don't know00:06
JayFmaking money off what other people don't know: the quinessential capitalist way00:07
mordredJayF: well ... it's actually been super good for a long time, but the number of people who know all the secrets was and remains ... small00:07
mordredyup00:07
mordredIf I cared more, I'd write some blogs posts - but I have to get really pissed off to re-engage with that part of my brain :)00:08
JayFwriting a blog telling someone how to do something is at least 100x harder than doing it yourself00:08
JayFlol00:08
clarkbmordred: ya I just remember sitting next to trove people saying things like you had to use xtrabackup00:08
JayFheading out, see you all at the ptg o/00:08
* mordred really should write up the "this is how to set up a 100% rock solid will never die DRBD failover pair for mysql that will give you more HA than you will ever need in 10 minutes"00:08
clarkbmordred: clearly I should read more betterer00:08
ianwhttp://people.redhat.com/~iwienand/status/ <- image build status00:09
*** krtaylor has quit IRC00:09
clarkbianw: soall our images are 17 days old?00:09
ianwwell, either that or i'm calculating wrong ... that was my next thing to figure out :)00:10
mordredthat doesn't seem great00:10
ianwthis is why i want to start monitoring it again :)00:10
mordredianw: did we have a dib issue?00:10
clarkbxenial 100 is 00:07:42:37 old00:10
clarkbwhich is 7 hours 42 minutes 37 seconds00:10
clarkbso I think maths somewhere might be wrong00:11
ianwhmm, the "age" is in microseconds?00:11
ianwmaybe not00:11
*** baoli has joined #openstack-infra00:11
*** andreww has quit IRC00:11
*** jogo has quit IRC00:12
*** Sukhdev has quit IRC00:12
*** jogo has joined #openstack-infra00:12
clarkbianw: what sort of tipped me off is both xenial and both trusty and so on have the same age00:13
clarkband that should be really hard for dib to do since it shouldn't run them more than 24 hours aport00:13
clarkber s/dib/nodepool-builder/00:13
clarkbs/more/less/ communication is hard00:13
*** baoli has quit IRC00:16
*** sdague has quit IRC00:16
ianwhmm, so the age is 148717173200:16
*** john-davidge has joined #openstack-infra00:16
ianwthat must be ms00:16
clarkbthat 413ish hours if milliseconds00:18
clarkbwhich doesn't seem right either00:18
ianwthat's where the 17 days is coming from00:18
clarkbah ok00:18
*** ihrachys has joined #openstack-infra00:18
ianwok, ns doesn't seem right either.  i'm just going have to look at the source i guess :)00:18
mordredShrews: ^^ what's the age of images?00:18
* mordred uses meat-based code search00:18
clarkbianw: its an epoch time00:20
ianwahh, that makes sense00:20
clarkbianw: from when image was built so you have to take the difference between epoch.now and then00:20
*** john-davidge has quit IRC00:21
*** Sukhdev has joined #openstack-infra00:21
clarkbso age is a derived thing we present to the user but storage on "disk" is epoch when state changed00:22
clarkbanyways simple enough once you know that :)00:22
ianwok, reload it and i think it's right00:23
clarkbya that looks right00:23
*** salv-orlando has joined #openstack-infra00:24
*** wolverineav has joined #openstack-infra00:24
*** Sukhdev has quit IRC00:25
*** Julien-zte has joined #openstack-infra00:28
*** rbrndt has quit IRC00:29
*** salv-orlando has quit IRC00:29
*** wolverineav has quit IRC00:33
*** hongbin has quit IRC00:35
*** abelur_ has joined #openstack-infra00:37
*** gouthamr has joined #openstack-infra00:39
*** abelur_ has quit IRC00:40
jeblairianw: any chance you can add that to nodepool itself?00:40
ianwjeblair: just writing the change description now :)00:40
jeblairianw: sweet :)00:41
fungiyeah, i figured that was just the mock-up/poc00:41
*** EricGonc_ has quit IRC00:43
*** mriedem has joined #openstack-infra00:44
*** wolverineav has joined #openstack-infra00:45
*** armax has joined #openstack-infra00:45
*** ihrachys has quit IRC00:47
openstackgerritJames E. Blair proposed openstack-infra/zuul feature/zuulv3: Add support for roles in zuul  https://review.openstack.org/43517900:48
*** wolverineav has quit IRC00:49
clarkbfungi: up to 187GB free on logs volume00:52
fungislowly but surely00:53
*** krtaylor has joined #openstack-infra00:53
clarkbJust need the multinode job on mriedem's change to come back so I can approve it then go grocery shopping00:54
*** liusheng has joined #openstack-infra00:57
*** rbuzatu has joined #openstack-infra01:01
*** armax has quit IRC01:03
*** wolverineav has joined #openstack-infra01:03
*** armax has joined #openstack-infra01:03
*** Hal has quit IRC01:04
*** wolverineav has quit IRC01:08
*** liujiong has joined #openstack-infra01:09
*** rajinir_ has quit IRC01:10
*** ZZelle_ has quit IRC01:11
clarkbthe change seems to have had the desired effect and qemu was set in both localrcs on the neutron multinode job01:15
clarkbI went ahead and approve it01:15
fungiperfect01:16
*** thorst_afk has joined #openstack-infra01:17
*** jamielennox is now known as jamielennox|away01:18
*** tphummel has quit IRC01:18
*** Hal has joined #openstack-infra01:20
mriedemthanks guys01:21
*** admcleod has quit IRC01:21
*** camunoz has quit IRC01:21
*** mtanino has quit IRC01:22
openstackgerritMerged openstack-infra/tripleo-ci master: Deploy novajoin in the TLS-everywhere job  https://review.openstack.org/42748101:22
*** baoli has joined #openstack-infra01:22
openstackgerritMerged openstack-infra/tripleo-ci master: Fix quickstart issues  https://review.openstack.org/42545901:22
openstackgerritIan Wienand proposed openstack-infra/nodepool master: Add a HTML status page  https://review.openstack.org/43518501:23
*** jamielennox|away is now known as jamielennox01:25
*** baoli has quit IRC01:27
*** dave-mccowan has joined #openstack-infra01:27
*** wolverineav has joined #openstack-infra01:27
*** yanyanhu has joined #openstack-infra01:28
*** sdake_ has joined #openstack-infra01:28
*** Julien-z_ has joined #openstack-infra01:29
*** Julien-zte has quit IRC01:30
*** sdake has quit IRC01:30
*** wolverineav has quit IRC01:31
*** admcleod has joined #openstack-infra01:33
*** camunoz has joined #openstack-infra01:34
*** Julien-zte has joined #openstack-infra01:34
*** Julien-z_ has quit IRC01:36
*** rbuzatu has quit IRC01:36
*** rfolco has joined #openstack-infra01:36
*** rfolco has quit IRC01:36
*** liuhaijie has joined #openstack-infra01:36
*** liuhaijie has quit IRC01:38
*** jcoufal has quit IRC01:39
*** liuhaijie has joined #openstack-infra01:39
*** thorst_afk has quit IRC01:41
*** baoli has joined #openstack-infra01:44
*** baoli has quit IRC01:44
*** baoli has joined #openstack-infra01:45
*** neiljerram has joined #openstack-infra01:46
neiljerramDid the gate just change so as to disallow external network access, by any chance?01:47
*** liuhaijie has quit IRC01:47
clarkbneiljerram: no but osic has had some issues01:48
clarkbhow recent are the problems you are seeing?01:48
neiljerramclarkb, Just today (UK time), I think; but possibly yesterday also.01:49
*** wolverineav has joined #openstack-infra01:49
clarkbok osic thought they had it sorted out but if it persists let us know01:50
neiljerramclarkb, There's a step in my test that adds a PPA and then installs some packages from that, and that has started failing like this:01:50
openstackgerritJuan Antonio Osorio Robles proposed openstack-infra/tripleo-ci master: DO NOT MERGE - testing TLS everywhere job  https://review.openstack.org/42686301:50
neiljerram2017-02-17 00:48:47.960 | ++ /opt/stack/new/networking-calico/devstack/plugin.sh:source:96 :   sudo apt-add-repository -y ppa:project-calico/calico-2.001:50
neiljerram2017-02-17 00:48:54.484 | Cannot add PPA: 'ppa:~project-calico/ubuntu/calico-2.0'.01:50
neiljerram2017-02-17 00:48:54.484 | ERROR: '~project-calico' user or team does not exist.01:50
neiljerramclarkb, Does that sound like the osic issue?01:51
clarkbif it ran in osic yes01:51
neiljerramFor my education, what is osic?01:52
mordredneiljerram: sudo add-apt-repository ppa:project-calico/calico-2.001:52
mordredok - just making sure - that's the same thing01:52
mordredseems to work for me locally01:52
clarkbneiljerram: openstack innovation center. One of our cloud providers01:52
*** camunoz has quit IRC01:52
neiljerramclarkb, thanks, I'll look further at the log, to see01:52
neiljerrammordred, the invocation doesn't have the ~; it's just the output that has that.01:53
mordredyah01:53
*** wolverineav has quit IRC01:54
neiljerramclarkb, ah yes, the log has: "ansible_fqdn: ubuntu-xenial-osic-cloud1-disk-7350617"01:54
neiljerramclarkb, I shall cross my fingers and try again tomorrow!01:55
*** thorst_afk has joined #openstack-infra01:56
*** Goneri has quit IRC02:02
*** abelur_ has joined #openstack-infra02:04
*** baoli has quit IRC02:04
*** winggundamth has joined #openstack-infra02:04
*** camunoz has joined #openstack-infra02:05
*** AJaeger has quit IRC02:08
*** dfflanders has quit IRC02:09
*** dfflanders has joined #openstack-infra02:10
*** dhinesh has quit IRC02:12
*** tuanluong has joined #openstack-infra02:16
*** john-davidge has joined #openstack-infra02:17
*** zz_dimtruck is now known as dimtruck02:19
*** sflanigan is now known as sflanigan_afk02:21
*** neiljerram has quit IRC02:21
*** john-davidge has quit IRC02:22
*** winggundamth has quit IRC02:23
*** tqtran has quit IRC02:24
*** winggundamth has joined #openstack-infra02:25
*** salv-orlando has joined #openstack-infra02:26
*** ijw has quit IRC02:26
*** baoli has joined #openstack-infra02:27
*** armax has quit IRC02:29
*** ijw has joined #openstack-infra02:30
*** salv-orlando has quit IRC02:30
*** guchihiro has joined #openstack-infra02:31
*** ijw has quit IRC02:31
*** EricGonczer_ has joined #openstack-infra02:32
*** rbuzatu has joined #openstack-infra02:32
*** ianychoi has quit IRC02:34
ianwmtreinish: ok, i think i'm getting it.  as a subunit2sql plugin, it will get called from logstash, but essentially it will need its own table to store in?02:35
*** ianychoi has joined #openstack-infra02:38
*** EricGonc_ has joined #openstack-infra02:39
*** tphummel has joined #openstack-infra02:40
*** EricGonczer_ has quit IRC02:40
*** beslemon has joined #openstack-infra02:44
*** sflanigan_afk is now known as sflanigan02:44
*** wolverineav has joined #openstack-infra02:45
*** Apoorva has quit IRC02:49
*** thorst_afk has quit IRC02:50
*** abelur_ has quit IRC02:52
*** guchihiro has quit IRC02:52
*** wolverineav has quit IRC02:52
openstackgerritIan Wienand proposed openstack-infra/nodepool master: Add image-list.json webapp target  https://review.openstack.org/43520102:53
yamamotoanything changed recently wrt osic?  https://bugs.launchpad.net/networking-midonet/+bug/166551402:53
openstackLaunchpad bug 1665514 in networking-midonet "Unable to connect to builds.midonet.org:http:" [Critical,New] - Assigned to YAMAMOTO Takashi (yamamoto)02:53
openstackgerritIan Wienand proposed openstack-infra/nodepool master: Add a HTML status page  https://review.openstack.org/43518503:00
*** AJaeger has joined #openstack-infra03:00
*** Sukhdev has joined #openstack-infra03:02
*** thorst_afk has joined #openstack-infra03:03
*** EricGonc_ has quit IRC03:03
*** thorst_afk has quit IRC03:04
*** rbuzatu has quit IRC03:07
*** baoli has quit IRC03:12
*** beslemon_ has joined #openstack-infra03:19
*** beslemon has quit IRC03:19
openstackgerritEmilien Macchi proposed openstack-infra/tripleo-ci master: TEST: DONT RECHECK or REVIEW: periodic jobs  https://review.openstack.org/35921503:20
*** deepbook5broo has joined #openstack-infra03:28
*** deepbook5broo has left #openstack-infra03:28
*** pradk has quit IRC03:32
*** armax has joined #openstack-infra03:32
*** armax has quit IRC03:35
*** thorst_afk has joined #openstack-infra03:35
*** thorst_afk has quit IRC03:35
*** sdake_ has quit IRC03:36
*** sdake has joined #openstack-infra03:39
*** armax has joined #openstack-infra03:43
*** YorikSar has quit IRC03:43
*** hongbin has joined #openstack-infra03:43
*** YorikSar has joined #openstack-infra03:44
*** dave-mccowan has quit IRC03:47
*** nmathew has joined #openstack-infra03:49
*** david-lyle has quit IRC03:49
*** david-lyle has joined #openstack-infra03:50
openstackgerritMerged openstack-infra/devstack-gate master: Set LIBVIRT_TYPE in the subnode localrc  https://review.openstack.org/43515403:50
*** sdake_ has joined #openstack-infra03:53
*** YorikSar has quit IRC03:54
*** YorikSar has joined #openstack-infra03:54
*** Julien-zte has quit IRC03:54
*** sdake has quit IRC03:55
*** mdrabe has joined #openstack-infra03:57
openstackgerritIan Wienand proposed openstack-infra/puppet-nodepool master: Deploy contrib/status HTML page  https://review.openstack.org/43521803:57
*** adrian_otto has joined #openstack-infra03:58
*** Sukhdev has quit IRC04:00
*** tphummel has quit IRC04:03
*** rbuzatu has joined #openstack-infra04:03
*** links has joined #openstack-infra04:03
*** sbezverk has joined #openstack-infra04:11
*** aeng_ has joined #openstack-infra04:11
*** davidlenwell is now known as anytone04:12
*** anytone is now known as davidlenwell04:13
jheskethFYI I'm going to finish retiring liberty branches as per tonyb's request04:14
*** adrian_otto has quit IRC04:14
*** wolverineav has joined #openstack-infra04:16
*** nicolasbock has quit IRC04:17
*** john-davidge has joined #openstack-infra04:18
*** psachin has joined #openstack-infra04:18
*** wolverineav has quit IRC04:21
*** wolverineav has joined #openstack-infra04:22
*** hongbin has quit IRC04:22
*** john-davidge has quit IRC04:22
*** dimtruck is now known as zz_dimtruck04:24
*** gongysh has joined #openstack-infra04:26
*** salv-orlando has joined #openstack-infra04:26
*** wolverineav has quit IRC04:29
*** salv-orlando has quit IRC04:31
*** adrian_otto has joined #openstack-infra04:34
*** rbuzatu has quit IRC04:38
*** wolverineav has joined #openstack-infra04:38
*** gouthamr has quit IRC04:38
*** kzaitsev_mb has quit IRC04:40
openstackgerritMerged openstack-infra/project-config master: murano: Use normal python jobs  https://review.openstack.org/42905004:40
openstackgerritMerged openstack-infra/project-config master: Remove link to dead system-config/jenkins documentation  https://review.openstack.org/43511304:41
*** sree has joined #openstack-infra04:43
*** esikache1 has joined #openstack-infra04:43
openstackgerritMerged openstack-infra/project-config master: Be verbose when running npm commands  https://review.openstack.org/43506304:45
openstackgerritMerged openstack-infra/project-config master: tacker: Use normal python jobs  https://review.openstack.org/42986804:45
*** sree has quit IRC04:47
*** adriant has quit IRC04:48
*** tqtran has joined #openstack-infra04:51
*** wolverineav has quit IRC04:52
*** tphummel has joined #openstack-infra04:54
*** adrian_otto has quit IRC04:54
*** sdake_ has quit IRC04:54
*** tqtran has quit IRC04:55
*** thorst_afk has joined #openstack-infra04:57
*** nunchuck has quit IRC05:02
*** AJaeger has quit IRC05:02
*** thorst_afk has quit IRC05:02
openstackgerritMorgan Fainberg proposed openstack-infra/shade master: Require `complete_qs` if a query_string is in the mocked uri  https://review.openstack.org/43512705:05
openstackgerritMorgan Fainberg proposed openstack-infra/shade master: Convert use of .register_uri to .register_uris  https://review.openstack.org/43451605:05
openstackgerritMorgan Fainberg proposed openstack-infra/shade master: Convert test_object to use .register_uris  https://review.openstack.org/43453205:05
*** mdrabe has quit IRC05:05
openstackgerritMorgan Fainberg proposed openstack-infra/shade master: Convert test_object to use .register_uris  https://review.openstack.org/43453205:05
*** AJaeger has joined #openstack-infra05:06
*** beslemon has joined #openstack-infra05:08
*** beslemon_ has quit IRC05:08
*** rcernin has joined #openstack-infra05:08
*** gongysh has quit IRC05:10
*** wolverineav has joined #openstack-infra05:12
*** armax has quit IRC05:14
*** beslemon has quit IRC05:16
*** wolverineav has quit IRC05:17
*** rlandy|afk is now known as rlandy05:19
*** rlandy has quit IRC05:19
*** wolverineav has joined #openstack-infra05:23
*** hichihara has joined #openstack-infra05:26
*** wolverineav has quit IRC05:27
*** salv-orlando has joined #openstack-infra05:27
*** esikache1 has quit IRC05:31
*** salv-orlando has quit IRC05:31
*** faizy has joined #openstack-infra05:32
*** tlian has quit IRC05:32
*** rbuzatu has joined #openstack-infra05:34
*** kzaitsev_mb has joined #openstack-infra05:35
*** Julien-zte has joined #openstack-infra05:37
ianwi think we might have a problem with osic05:38
ianwnodepool@nodepool:~$ nodepool list | grep centos-7 | grep building | wc -l05:38
ianw12005:38
ianwnodepool@nodepool:~$ nodepool list | grep centos-7 | grep osic | wc -l05:38
ianw11705:38
ianwthere seems to be a real lack of centos-7 nodes ATM and they're all building in osic05:39
ianwi can see from console log they boot ok to a prompt, but seem to get no address05:39
ianwwe also have a revert out for disabling osic in -> https://review.openstack.org/#/c/435148/ , but afaict from the config file it is currently enabled ... maybe someone edited by hand?05:40
*** esikache1 has joined #openstack-infra05:40
*** kzaitsev_mb has quit IRC05:40
ianwthat must be a dupe ...05:41
openstackgerritMorgan Fainberg proposed openstack-infra/shade master: Convert test_object to use .register_uris  https://review.openstack.org/43453205:44
*** jbernard has quit IRC05:47
openstackgerritIan Wienand proposed openstack-infra/project-config master: Revert "Revert "Temporarily disable osic-cloud1""  https://review.openstack.org/43525005:48
ianwAJaeger / jhesketh / yolanda : i'm afraid i'm about to disappear for commitments here, but the gate seems backed up on centos-7 nodes that seem stuck building in OSIC.  ^ is about my best guess, and should redistribute the load?05:49
*** tqtran has joined #openstack-infra05:53
AJaegerianw: let's merge pabelanger's https://review.openstack.org/#/c/435148 ...05:54
*** gildub has quit IRC05:55
*** jbernard has joined #openstack-infra05:56
*** tqtran has quit IRC05:58
* AJaeger approves that for now05:58
*** yamahata has joined #openstack-infra06:00
openstackgerritMerged openstack-infra/project-config master: Revert "Revert "Temporarily disable osic-cloud1""  https://review.openstack.org/43514806:06
openstackgerritAndreas Jaeger proposed openstack-infra/project-config master: Remove now unused db jobs  https://review.openstack.org/43526006:07
*** camunoz has quit IRC06:08
*** yamamoto has quit IRC06:08
*** rbuzatu has quit IRC06:08
*** nadya has joined #openstack-infra06:10
*** amotoki has joined #openstack-infra06:12
*** gildub has joined #openstack-infra06:12
*** jcoufal has joined #openstack-infra06:15
*** zhipengh has joined #openstack-infra06:15
ianwAJaeger: I think that one was a dup of the other one i mentioned in a comment06:17
ianwcause osic is enabled06:17
*** john-davidge has joined #openstack-infra06:18
*** bhavik1 has joined #openstack-infra06:21
*** rcernin has quit IRC06:21
ianwahh, no, ok that's putting it back to 006:22
ianwtoo many reverts :)06:23
*** john-davidge has quit IRC06:23
*** nadya has quit IRC06:26
*** hichihara has quit IRC06:26
*** wolverineav has joined #openstack-infra06:31
zhipenghHi infra cores could you hel merge this patch https://review.openstack.org/#/c/421322/ ?06:32
*** bhavik1 has quit IRC06:32
*** gildub has quit IRC06:32
*** wolverineav has quit IRC06:36
*** yuanying has quit IRC06:37
*** kzaitsev_mb has joined #openstack-infra06:39
tonybjhesketh: Thanks.06:39
*** jcoufal has quit IRC06:45
*** pahuang has quit IRC06:46
*** eroux has joined #openstack-infra06:47
*** yamamoto has joined #openstack-infra06:47
*** flepied has joined #openstack-infra06:48
*** wolverineav has joined #openstack-infra06:54
*** wolverineav has quit IRC06:58
*** thorst_afk has joined #openstack-infra06:59
*** amotoki has quit IRC07:01
*** rcernin has joined #openstack-infra07:01
*** thorst_afk has quit IRC07:03
*** cshastri has joined #openstack-infra07:04
*** esikache1 has quit IRC07:04
*** eroux has quit IRC07:04
*** askb has quit IRC07:04
*** rbuzatu has joined #openstack-infra07:05
*** aeng_ has quit IRC07:05
*** amotoki has joined #openstack-infra07:08
*** esikache1 has joined #openstack-infra07:09
*** aeng_ has joined #openstack-infra07:11
*** tesseract has joined #openstack-infra07:11
*** esikache1 has quit IRC07:13
*** anilvenkata has joined #openstack-infra07:13
*** askb has joined #openstack-infra07:15
*** kzaitsev_mb has quit IRC07:19
*** aeng_ has quit IRC07:22
*** salv-orlando has joined #openstack-infra07:22
*** andreas_s has joined #openstack-infra07:25
openstackgerritAndreas Jaeger proposed openstack-infra/project-config master: Revert "Revert "Revert "Temporarily disable osic-cloud1"""  https://review.openstack.org/43527907:27
AJaegerianw: pushed up a revert for later ^07:27
*** markus_z has joined #openstack-infra07:29
*** vsaienk0 has quit IRC07:30
*** knangia has quit IRC07:30
*** zhipengh has quit IRC07:32
AJaegeryolanda: please put https://review.openstack.org/435260 on your review queue as well07:36
yolandahi good morning07:36
AJaegermorning!07:36
yolandaAJaeger, will you be at the ptg?07:37
*** vsaienk0 has joined #openstack-infra07:37
*** vsaienk0 has left #openstack-infra07:37
*** wolverineav has joined #openstack-infra07:38
*** rbuzatu has quit IRC07:39
*** gongysh has joined #openstack-infra07:41
openstackgerritMerged openstack-infra/project-config master: Add new project: vmware-vspc  https://review.openstack.org/43314407:42
*** admcleod has quit IRC07:43
*** wolverineav has quit IRC07:44
openstackgerritMerged openstack-infra/project-config master: Enable signature verification for barbican-tempest-plugin gate  https://review.openstack.org/43438407:47
*** jd__ has quit IRC07:47
AJaeger#status log osic-cloud disabled again, see https://review.openstack.org/435250 for some background07:47
openstackstatusAJaeger: finished logging07:48
AJaegeryolanda: no - travelling to Spain and Portugal for presenting instead ;(07:48
openstackgerritMerged openstack-infra/project-config master: Retiring python-wsmanclient: remove all jobs and set project read-only  https://review.openstack.org/43477907:50
yolandawhere in Spain?07:50
openstackgerritMerged openstack-infra/project-config master: Run amqp1 non-voting job only on master  https://review.openstack.org/43466207:50
AJaegerSevilla on Tuesday07:51
*** nadya has joined #openstack-infra07:52
yolandaoh, that's nice!07:52
yolandahave you ever been?07:52
*** kzaitsev_mb has joined #openstack-infra07:52
*** nadya has quit IRC07:53
*** nadya has joined #openstack-infra07:53
AJaegerNever ;( So, hope to find an hour ;) to walk around...07:53
*** sree has joined #openstack-infra07:54
AJaegeryolanda: could you +2A clarkb's devstack-gate change, please? https://review.openstack.org/#/c/417554/07:54
AJaegeryolanda: will you be in Atlanta?07:54
*** tqtran has joined #openstack-infra07:55
*** wolverineav has joined #openstack-infra07:57
*** saibarspeis has joined #openstack-infra07:57
yolandaAJaeger, yes07:57
yolandaflying on sunday07:57
*** shardy_afk is now known as shardy07:57
*** yuanying has joined #openstack-infra07:57
*** migi_ is now known as migi07:58
*** tqtran has quit IRC07:59
*** rossella_s has joined #openstack-infra07:59
AJaegeryolanda: cool! Enjoy it and safe travels!08:01
*** csomerville has quit IRC08:01
*** csomerville has joined #openstack-infra08:02
yolandathx08:02
*** wolverineav has quit IRC08:07
AJaegerodyssey4me: does ansible still have a liberty branch that we need to take care of in project-config?08:12
*** sree has quit IRC08:12
odyssey4meAJaeger nope08:12
odyssey4meI was going to clean those up when mitaka goes EOL08:12
odyssey4meunless, of course, andymccr did it first :p08:12
AJaeger;)08:12
* AJaeger does a small cleanup right now...08:12
odyssey4meah ok, go ahead08:12
openstackgerritOpenStack Proposal Bot proposed openstack-infra/project-config master: Normalize projects.yaml  https://review.openstack.org/43529808:13
*** sree has joined #openstack-infra08:14
*** fandi has joined #openstack-infra08:14
openstackgerritAndreas Jaeger proposed openstack-infra/project-config master: Remove liberty branch conditions from some projects  https://review.openstack.org/43529908:16
AJaegerodyssey4me, andymccr , could you review this, please ? ^08:16
odyssey4meLGTM08:17
*** yamamoto has quit IRC08:18
AJaegerthanks08:18
*** aviau has quit IRC08:19
*** john-davidge has joined #openstack-infra08:19
*** aviau has joined #openstack-infra08:19
*** admcleod has joined #openstack-infra08:21
*** pcaruana has joined #openstack-infra08:22
*** john-davidge has quit IRC08:24
*** ijw has joined #openstack-infra08:27
odyssey4meAJaeger argh, the combination of filters in zuul layout for OSA are giving me a headache08:27
AJaegeryolanda: please put https://review.openstack.org/435299 on your review queue for later. Also https://review.openstack.org/#/c/433363/ is ready now IMHO.08:28
*** rbuzatu has joined #openstack-infra08:29
AJaegerodyssey4me: same to me ;(08:29
AJaegerI just downloaded your change...08:29
AJaegerlet me stare at it...08:29
*** sree has quit IRC08:29
odyssey4methanks AJaeger08:29
AJaegerodyssey4me: do you have an example job that runs jobs it shouldn't?08:30
odyssey4meit's all made complicated by the simple idea that we need to run trusty *func* jobs for stable/newton08:30
AJaegerexample change I mean08:30
odyssey4meyou mean a change that's not yet merged?08:30
odyssey4meie a review to test with08:30
AJaegerno, a change that showed the wrong behaviour - merged or not08:30
*** yamamoto has joined #openstack-infra08:31
odyssey4melet me dig one up08:31
*** wolverineav has joined #openstack-infra08:32
*** ijw has quit IRC08:32
*** rbuzatu has quit IRC08:33
odyssey4meAJaeger here's an example: https://review.openstack.org/#/q/Ibb689431f288190f3a8451b52cd77a4167d6945c,n,z08:33
odyssey4methe same change id across multiple branches08:34
odyssey4mefor master/ocata the trusty jobs should not be running08:34
odyssey4mefor newton they should be running08:34
*** sflanigan has quit IRC08:35
odyssey4meif you're confused about why they're passing, it's because we implemented a workaround in our tests to just make it pass so that we could continue development08:35
odyssey4methere's a bunch of job names with whether they should or should not run in the commit message for https://review.openstack.org/43150108:35
AJaegerodyssey4me: let me propose something... I have an idea...08:36
*** wolverineav has quit IRC08:37
*** admcleod has quit IRC08:37
*** admcleod has joined #openstack-infra08:38
*** admcleod has joined #openstack-infra08:38
*** florianf has joined #openstack-infra08:38
openstackgerritRadoslav Gerganov proposed openstack-infra/project-config master: Add non-voting py35 job for nova-mksproxy  https://review.openstack.org/43493208:40
*** florianf has quit IRC08:42
*** yamamoto has quit IRC08:44
*** ccamacho has joined #openstack-infra08:44
ianwAJaeger: yeah, i think the building nodes is settling -> http://graphite.openstack.org/render/?from=-24hours&height=180&until=now&width=334&bgcolor=ffffff&fgcolor=000000&areaMode=stacked&target=color(alias(sumSeries(stats.gauges.nodepool.nodes.building),%20%27Building%27),%20%27ffbf52%27)&target=color(alias(sumSeries(stats.gauges.nodepool.nodes.ready),%20%27Available%27),%20%2700c868%27)&target=color(alias(sumSeries(stats.gauges.nodepool.08:44
ianwnodes.used),%20%27In%20Use%27),%20%276464ff%27)&target=color(alias(sumSeries(stats.gauges.nodepool.nodes.delete),%20%27Deleting%27),%20%27c864ff%27)&title=Test%20Nodes&_t=0.3630118636684503#148732106216608:44
ianwoh, wow, sorry that was way longer than i thought it was :)08:45
*** jordanP has joined #openstack-infra08:49
*** florianf has joined #openstack-infra08:49
openstackgerritAndreas Jaeger proposed openstack-infra/project-config master: OSA: Close trusty job leaks for Ocata onwards  https://review.openstack.org/43150108:50
AJaegerianw: great08:50
*** sree has joined #openstack-infra08:50
AJaegerodyssey4me: I'm testing 431501 now, please review it.08:50
AJaegerianw: http://grafana.openstack.org/dashboard/db/nodepool looks good - your line is far too long for me ;)08:51
*** florianf has quit IRC08:51
*** jpich has joined #openstack-infra08:52
*** admcleod has quit IRC08:52
AJaegerodyssey4me: change is wrong, I've overlooked something...08:53
AJaegerodyssey4me: why should gate-openstack-ansible-os_keystone-ansible-func-ubuntu-trusty run on newton but not mitaka - while gate-openstack-ansible-rabbitmq_server-ansible-func-ubuntu-trusty sould run on both?08:55
*** ssbarnea has joined #openstack-infra08:55
odyssey4meAJaeger hmm, I must have given wrong informaiton - those should both run on both08:55
*** wolverineav has joined #openstack-infra08:56
openstackgerritAndreas Jaeger proposed openstack-infra/project-config master: OSA: Close trusty job leaks for Ocata onwards  https://review.openstack.org/43150108:56
AJaegerok, then let's try this ^08:56
AJaegerIf that's correct, I let you update the commit message ;)08:56
*** sree has quit IRC08:57
odyssey4meok, it looks like some of the other jobs only meant for ocata onwards will slip through those cracks08:57
odyssey4mefor mitaka only the 'func' scenario should run08:57
odyssey4meactually, only the aio|func scenario's08:58
*** pramodrj07 has joined #openstack-infra08:58
AJaegertrusty or xenial?08:58
odyssey4mein the change, see 3313 on the left08:58
AJaegerand that is *later*, so overrides what  I do ;)08:58
AJaegerodyssey4me: Assume the algorithm works as follows:08:59
AJaeger1) For each job do08:59
*** thorst_afk has joined #openstack-infra08:59
AJaeger   for each regex do08:59
openstackgerritMatthew Oliver proposed openstack-infra/project-config master: Stop swifts func-post-as-copy job running on stable  https://review.openstack.org/43466408:59
*** zzzeek has quit IRC09:00
AJaeger  Does the regex match? Assign (overwrite!) branch condition09:00
AJaeger  done09:00
AJaegerdone09:00
*** zzzeek has joined #openstack-infra09:00
AJaegerso, an earlier condition will be overwritten by a later one. That's why  I start with very generic - too generic ones - and then fix them with specific ones09:00
AJaegerodyssey4me: I'm waiting for tox -e zuul output to check that it's working as expected now09:01
*** MasterOfBugs has quit IRC09:01
*** wolverineav has quit IRC09:01
openstackgerritSteven Hardy proposed openstack-infra/tripleo-ci master: remove test-environments/multinode_major_upgrade.yaml  https://review.openstack.org/42637509:01
odyssey4meAJaeger I'm trying to find the part in the commit message which you think isn't right?09:01
*** thorst_afk has quit IRC09:04
AJaegerhttp://paste.openstack.org/show/599346/09:05
*** wolverineav has joined #openstack-infra09:06
AJaegerodyssey4me: I'll fix - found a typo...09:08
openstackgerritMatthew Oliver proposed openstack-infra/project-config master: Stop swifts func-post-as-copy job running on stable  https://review.openstack.org/43466409:08
*** bhavik1 has joined #openstack-infra09:09
*** esikache1 has joined #openstack-infra09:09
openstackgerritAndreas Jaeger proposed openstack-infra/project-config master: OSA: Close trusty job leaks for Ocata onwards  https://review.openstack.org/43150109:11
AJaegernext try, odyssey4me ^09:12
odyssey4meok, waiting for the job result to verify :)09:13
odyssey4methanks AJaeger09:13
*** jpich has quit IRC09:14
AJaegerso do I ;) But please review the logic in my change. Does my pseudo algorithm above help?09:15
*** dtantsur|afk is now known as dtantsur09:16
*** chem has joined #openstack-infra09:17
AJaegerodyssey4me: http://paste.openstack.org/show/599349/09:18
AJaeger(ignore duplicates, first is check, then gate, then post queue - I did a simple grep for trusty09:19
*** lucas-afk is now known as lucasagomes09:20
*** gildub has joined #openstack-infra09:21
AJaegerodyssey4me: http://paste.openstack.org/show/599350/ for all jobs for os_keystone09:21
AJaegerodyssey4me: please double check ;)09:22
odyssey4methat does *seem* right09:22
odyssey4methat said, I do find that I often think it's right and then it turns out to expose some other gap09:23
odyssey4mebut I think this should help us plug the gaps in the right way09:23
AJaegerodyssey4me: That's why I think it's better to be not too clever - start with a catchall and cleanup.09:23
odyssey4meyep, agreed09:23
AJaegerOr create a condition for each and every job...09:24
odyssey4mein the platform clean up for mitaka, do we need to also clean up the xenial jobs?09:24
AJaegerodyssey4me: so, for keystone this looks as mentioned in commit message...09:24
odyssey4meor is there something elsewhere that prevents those running?09:24
AJaegerthere's a global regex that runs xenial only on mitaka or newer09:25
AJaegeron newton and newer I mean09:25
odyssey4meok good09:25
AJaegershould be fine - let's see what surprises we get ;)09:25
odyssey4methere may be some other functional tests which we may have to clean up, but I'll find them and plug them as we go09:26
AJaegerodyssey4me, might be best you and andymccr double check my edits and both +1...09:26
odyssey4meat least now the pattern is easier to follow09:26
AJaegerglad to hear09:26
*** jaosorior has joined #openstack-infra09:27
*** jpich has joined #openstack-infra09:29
*** rbuzatu has joined #openstack-infra09:29
*** kzaitsev_mb has quit IRC09:30
*** martinkopec has joined #openstack-infra09:30
openstackgerritTobias Henkel proposed openstack-infra/zuul master: Enhance mutex to counting semaphore  https://review.openstack.org/38652009:30
*** AJaeger has quit IRC09:31
*** AJaeger has joined #openstack-infra09:34
odyssey4meAJaeger ok, I found an error in my commit message test set: https://review.openstack.org/#/c/431501/5//COMMIT_MSG09:36
odyssey4meIt looks like those still run, which will be a problem09:36
*** Julien-zte has quit IRC09:38
*** neiljerram has joined #openstack-infra09:39
*** Jeffrey4l has quit IRC09:42
AJaegerodyssey4me: but that means that gate-openstack-ansible-os_keystone-ansible-uw_apache-ubuntu-trusty should not run at all and could be removed. Correect?09:43
odyssey4meAJaeger the trusty one yes09:44
AJaegeryes, the trusty one.09:44
odyssey4methe xenial one must run for newton09:44
AJaegerit does09:44
*** yamamoto has joined #openstack-infra09:44
AJaegerSo, let's remove gate-openstack-ansible-os_keystone-ansible-uw_nginx-ubuntu-trusty and the uw_apache...trusty09:44
odyssey4mewell, we should remove anything with the pattern09:45
odyssey4megate-openstack-ansible-.*-ansible-(?!(aio|func))-ubuntu-trusty09:45
odyssey4mesurely?09:45
odyssey4mebasically any functional test job on trusty that's not 'aio' or 'func'09:46
openstackgerritMerged openstack-infra/devstack-gate master: Add tox target for python3 test env  https://review.openstack.org/41755409:46
AJaegerif those will neither run on mitaka nor or newton, then remove them...09:46
AJaegermight be best to make this as separate change first09:46
odyssey4methose must run on newton09:47
odyssey4methat's why line 3313 was there09:47
*** matrohon has joined #openstack-infra09:48
AJaegerodyssey4me: let's add a specific regex for these jobs:09:48
openstackgerritJakub Libosvar proposed openstack-infra/project-config master: neutron: Run in-tree Neutron tempest tests with concurrency 2  https://review.openstack.org/43486609:48
AJaegergate-openstack-ansible-.*-ansible-uw_nginx-ubuntu-trusty - branch: newton or something like that.09:49
*** wolverineav has quit IRC09:49
*** yamamoto has quit IRC09:49
AJaegerLet's add a few *positive* regexes to handle these. Are you going to amend?09:50
*** denisra has joined #openstack-infra09:50
odyssey4meAJaeger the trouble is that there are loads of these, so it seems a little impractical to add an extra line for each one09:51
*** yamahata has quit IRC09:51
odyssey4mebut if that's more maintainable, then that's OK with me09:51
AJaegerodyssey4me: I think it's better maintainable. The negative tests let us to this mess...09:51
odyssey4meyeah09:52
AJaegerso, over to you for the change, please ;)09:52
odyssey4mewell, I'm not sure I understand how to do a positive regex to prevent the job running on mitaka?09:53
AJaegerLike above:09:53
odyssey4meoh, you mean a regex for the job which says 'this job must only run for newton, ocata, master ?09:53
AJaeger - name: gate-openstack-ansible-.*-ansible-uw_nginx-ubuntu-trusty09:54
AJaeger    branch: stable/newton09:54
odyssey4meoh I see09:54
odyssey4methat wil then restrict it to only running on the newton branch09:54
AJaegerIt's a trusty job, you don#t want it on ocata! The xenial will be on ocata!09:54
odyssey4meyep, I think I understand now - let me patch that up09:54
*** Jeffrey4l has joined #openstack-infra09:55
AJaegeradd a few comments as well ;)09:55
*** tqtran has joined #openstack-infra09:56
*** Jeffrey4l has quit IRC09:56
*** amotoki has quit IRC09:56
*** Jeffrey4l has joined #openstack-infra09:57
*** nadya has quit IRC09:59
*** tqtran has quit IRC10:00
*** yanyanhu has quit IRC10:01
*** ssbarnea has quit IRC10:02
*** tuanluong has quit IRC10:03
*** ssbarnea has joined #openstack-infra10:03
*** rbuzatu has quit IRC10:04
AJaegermordred: jeepby failed, https://github.com/openstack/vmware-vspc is not set up but https://git.openstack.org/cgit/openstack/vmware-vspc looks fine.10:07
*** jd__ has joined #openstack-infra10:08
*** derekh has joined #openstack-infra10:08
*** sambetts|afk is now known as sambetts10:11
*** faizy has quit IRC10:12
*** wolverineav has joined #openstack-infra10:13
*** faizy has joined #openstack-infra10:13
*** gongysh has quit IRC10:13
*** wolverineav has quit IRC10:13
*** wolverineav has joined #openstack-infra10:13
*** liujiong has quit IRC10:15
*** wolverineav has quit IRC10:15
openstackgerritJesse Pretorius (odyssey4me) proposed openstack-infra/project-config master: OSA: Close trusty job leaks for Ocata onwards  https://review.openstack.org/43150110:16
*** yamamoto has joined #openstack-infra10:17
*** Jeffrey4l has quit IRC10:18
*** Jeffrey4l has joined #openstack-infra10:19
AJaegerodyssey4me: you have one regex that matches both xenial and trusty - and for those I'm always suspicious ;)10:19
ssbarneahi guys! is anyone aware about a bug where the API always returns 0 for current used floatingips? openstack limits show --absolute | grep FloatingIps10:19
AJaegerRemove the trusty jobs as I commented and it should be fine10:19
*** john-davidge has joined #openstack-infra10:20
AJaegerssbarnea: better ask on #openstack-dev. And keep in mind that we're not only male here - guys can be seens as addressing males only.10:20
*** wolverineav has joined #openstack-infra10:20
odyssey4meAJaeger replied in review - it's intentional to remove those jobs regardless of platform for anything newton and below10:21
ssbarneaAJaeger: thanks! yep better to drop it to avoid confusions :)10:22
*** chem has quit IRC10:22
*** chem has joined #openstack-infra10:23
*** martinkopec has quit IRC10:23
*** bhavik1 has quit IRC10:24
*** esikache1 has quit IRC10:24
*** wolverineav has quit IRC10:25
*** john-davidge has quit IRC10:25
*** yamamoto has quit IRC10:25
*** martinkopec has joined #openstack-infra10:26
*** AJaeger has quit IRC10:27
*** sflanigan has joined #openstack-infra10:28
*** sflanigan has joined #openstack-infra10:28
*** Ahharu has joined #openstack-infra10:28
AhharuHello! I am having trouble to delete the branch and tag '9.2.0' in the fuel-plugin-midonet project I am working on , https://review.openstack.org/#/q/project:openstack/fuel-plugin-midonet since it appears I or anyone in Midokura org has rights to do so10:29
AhharuCan you help?10:29
*** jaosorior has quit IRC10:29
*** ldnunes has joined #openstack-infra10:30
*** jaosorior has joined #openstack-infra10:30
*** markus_z has quit IRC10:32
*** AJaeger has joined #openstack-infra10:32
*** nadya has joined #openstack-infra10:33
AJaegerodyssey4me: I meant to really delete the trusty job, you introduced again a leakage for master of a trusty one. See the paste I sent10:33
*** rbuzatu has joined #openstack-infra10:35
openstackgerritJesse Pretorius (odyssey4me) proposed openstack-infra/project-config master: OSA: Close trusty job leaks for Ocata onwards  https://review.openstack.org/43150110:35
odyssey4meAJaeger ah, missed that - thanks10:36
odyssey4mefixed10:36
*** rfolco has joined #openstack-infra10:36
AhharuAJaeger , probably you can help on this one10:37
AhharuI am having trouble to delete the branch and tag '9.2.0' in the fuel-plugin-midonet project I am working on , https://review.openstack.org/#/q/project:openstack/fuel-plugin-midonet since it appears I or anyone in Midokura org has rights to do so10:37
*** nadya has quit IRC10:38
openstackgerritSteven Hardy proposed openstack-infra/tripleo-ci master: remove test-environments/multinode_major_upgrade.yaml  https://review.openstack.org/42637510:39
openstackgerritSteven Hardy proposed openstack-infra/tripleo-ci master: upgrades: switch 'newton' to $UPGRADE_RELEASE  https://review.openstack.org/43320910:39
*** gkibardin has joined #openstack-infra10:39
gkibardinHi everybody10:39
gkibardinCould somebody please take a look why gate tests fails here https://review.openstack.org/#/c/430382/ ?10:40
gkibardinThis one http://logs.openstack.org/82/430382/3/gate/gate-fuel-web-python27-ubuntu-trusty/7bb9296/console.html10:41
gkibardinWhile the same test here http://logs.openstack.org/82/430382/3/check/gate-fuel-web-python27-ubuntu-trusty/8c5d20f/console.html is ok10:41
*** tosky has joined #openstack-infra10:41
*** kzaitsev_mb has joined #openstack-infra10:44
*** markus_z has joined #openstack-infra10:46
*** wolverineav has joined #openstack-infra10:48
openstackgerritJuan Antonio Osorio Robles proposed openstack-infra/tripleo-ci master: DO NOT MERGE - testing TLS everywhere job with RabbitMQ  https://review.openstack.org/42630710:48
*** pblaho has joined #openstack-infra10:49
*** nadya has joined #openstack-infra10:50
*** sdague has joined #openstack-infra10:50
*** khappone has quit IRC10:51
*** jaosorior has quit IRC10:52
*** wolverineav has quit IRC10:52
*** rossella_s has quit IRC10:53
*** jaosorior has joined #openstack-infra10:53
*** khappone has joined #openstack-infra10:53
AJaegerAhharu: we do not hand out branch deletion permissions, an admin needs to help you. Best come back during US time - I'm not an admin and nobody else is around10:54
*** rossella_s has joined #openstack-infra10:54
AhharuAJaeger: will try then10:55
AJaegergkibardin: nothing looks suspicious to an infra problem on that one.10:55
AJaegergkibardin: better ask in fuel channel10:56
*** wolverineav has joined #openstack-infra10:56
*** esikache1 has joined #openstack-infra10:56
*** john-davidge has joined #openstack-infra10:57
*** wolverineav has quit IRC11:00
gkibardinAJaeger: fuel guys have sent me here :)11:01
*** rbuzatu has quit IRC11:01
*** thorst_afk has joined #openstack-infra11:01
*** rbuzatu has joined #openstack-infra11:02
gkibardinThe suspicious thing here is the same test, on the same code fails and passes on enviroments which supposed to be identical11:02
*** rbuzatu_ has joined #openstack-infra11:03
*** martinkopec has quit IRC11:04
*** rbuzatu__ has joined #openstack-infra11:04
*** thorst_afk has quit IRC11:05
*** esikache1 has quit IRC11:05
*** martinkopec_ has joined #openstack-infra11:05
*** rbuzat___ has joined #openstack-infra11:06
*** rbuzatu has quit IRC11:06
*** rbuza____ has joined #openstack-infra11:06
*** rbuzatu has joined #openstack-infra11:07
*** rbuzatu_ has quit IRC11:08
*** rbuzatu__ has quit IRC11:09
*** rbuzat___ has quit IRC11:10
*** winggundamth has quit IRC11:10
*** rbuza____ has quit IRC11:11
*** rbuzatu has quit IRC11:12
*** wolverineav has joined #openstack-infra11:13
AJaegergkibardin: Nothing looks suspicious there to me, I cannot help further.11:15
*** Ahharu has quit IRC11:16
*** esikache1 has joined #openstack-infra11:16
*** sdake has joined #openstack-infra11:18
*** wolverineav has quit IRC11:18
*** nicolasbock has joined #openstack-infra11:19
*** AJaeger has quit IRC11:20
*** AJaeger has joined #openstack-infra11:20
*** esikache1 has quit IRC11:21
openstackgerritJakub Libosvar proposed openstack-infra/project-config master: neutron: Move linuxbridge Neutron scenario to -nv check queue  https://review.openstack.org/43538811:23
gkibardinAJaeger: do you mean check and gate environments are so different so one test may constantly pass on check and fail on gate?11:24
*** esikache1 has joined #openstack-infra11:26
openstackgerritVasyl Saienko proposed openstack-infra/project-config master: Made ironic-multinode job voting for Queens+  https://review.openstack.org/43224611:31
AJaegergkibardin: the environment is the same. But you run the job at different times, so new packages might get released (unless you use constraitns to freeze package versions), different cloud providers are used etc.11:31
AJaegergkibardin: we had some problems with osic cloud yesterday but your job was run in rax.11:32
*** e0ne has joined #openstack-infra11:32
AJaegergkibardin: you can add a  "recheck" comment and see whether it was a random failure. If not, you have to compare the logs and figure out what has changed ;(11:32
gkibardinAJaeger: I've just checked tox logs - the set of packages is the same, the code is also the same11:32
gkibardinI've did it twice11:33
*** esikache1 has quit IRC11:33
AJaegerpackage versions as well the same?11:33
gkibardinyes11:33
gkibardinboth times check passed and gate failed11:33
gkibardinwait, not exactly11:34
gkibardinit just stated that the requirement is satisfied11:34
gkibardinit doesn't necessary mean installed packages have the same version11:34
AJaegergkibardin: sorry, I don't have time to help further - hope others can do so.11:35
gkibardincan we just drop venv to ensure everything is similar?11:35
AJaegervenv?11:35
gkibardinAJaeger: ok, thank you anyway11:35
*** yolanda has quit IRC11:36
gkibardinAJaeger: where the tests are run11:36
*** yamamoto has joined #openstack-infra11:37
*** yamamoto has quit IRC11:37
openstackgerritOksana Voshchana proposed openstack-infra/devstack-gate master: TEST for multinode grenade devstack  https://review.openstack.org/39667911:38
*** jkilpatr has quit IRC11:39
*** wolverineav has joined #openstack-infra11:39
*** yolanda has joined #openstack-infra11:39
*** markus_z has quit IRC11:39
*** wolverineav has quit IRC11:43
*** rhallisey has joined #openstack-infra11:47
*** nmathew has quit IRC11:48
*** sdake has quit IRC11:49
*** sdake has joined #openstack-infra11:49
*** salv-orlando has quit IRC11:50
*** esikache1 has joined #openstack-infra11:50
*** markus_z has joined #openstack-infra11:54
*** esikache1 has quit IRC11:55
openstackgerritSteven Hardy proposed openstack-infra/tripleo-ci master: remove test-environments/multinode_major_upgrade.yaml  https://review.openstack.org/42637511:55
openstackgerritSteven Hardy proposed openstack-infra/tripleo-ci master: upgrades: switch 'newton' to $UPGRADE_RELEASE  https://review.openstack.org/43320911:55
*** esikache1 has joined #openstack-infra11:59
*** askb has quit IRC12:00
openstackgerritLee Yarwood proposed openstack-infra/project-config master: Layout: Ensure dsvm-full-devstack-plugin-ceph is non-voting for all  https://review.openstack.org/43540612:00
*** jkilpatr has joined #openstack-infra12:01
*** gkibardin has quit IRC12:07
*** gildub has quit IRC12:07
*** rbuzatu has joined #openstack-infra12:08
*** jkilpatr has quit IRC12:09
*** ansiwen_ is now known as ansiwen12:10
*** dave-mccowan has joined #openstack-infra12:13
*** fandi has quit IRC12:14
*** mhickey has joined #openstack-infra12:14
*** jaosorior has quit IRC12:16
lyarwoodAJaeger: just updated https://review.openstack.org/435406 with the example, sorry about that!12:17
lyarwoodAJaeger: ah, saw your comment, thanks!12:17
AJaegerlyarwood: change is still wrong ;)12:17
AJaegeryour change would force the job to run on those two branches that we currnetly  blacklist.12:18
AJaegerThe conditions really mean: Run it on branches matching these conditions only. And when it's run, it's non-voting.12:18
*** jkilpatr has joined #openstack-infra12:22
*** shardy is now known as shardy_lunch12:23
lyarwoodAJaeger: so I need to add a branch line limiting gate-tempest-dsvm-full-devstack-plugin-ceph-trusty to stable/mitaka and driverfixes right?12:24
*** nmathew has joined #openstack-infra12:24
*** nmathew has quit IRC12:25
AJaegerlyarwood: no branch condition needed at all, just like I put it in the ocmment.12:25
AJaegerwe have a global branch condition for trusty that is just fine12:26
lyarwoodah!12:26
lyarwoodAJaeger: thanks12:26
openstackgerritLee Yarwood proposed openstack-infra/project-config master: Layout: Make gate-tempest-dsvm-full-devstack-plugin-ceph-trusty non-voting  https://review.openstack.org/43540612:27
*** yamamoto has joined #openstack-infra12:27
AJaegerlyarwood: fix the typo and I'm happy ;)12:28
*** ociuhandu has joined #openstack-infra12:29
lyarwoodAJaeger: ah, that was from your example :D12:29
lyarwood1 sec12:29
openstackgerritLee Yarwood proposed openstack-infra/project-config master: Layout: Make gate-tempest-dsvm-full-devstack-plugin-ceph-trusty non-voting  https://review.openstack.org/43540612:30
*** jaosorior has joined #openstack-infra12:30
AJaegersorry ;(12:30
lyarwoodAJaeger: hehe np np12:31
AJaegerlyarwood: fine now, +212:31
lyarwoodAJaeger: thanks again12:32
lyarwoodsdague: ^ if you wouldn't mind hitting that again I'll take care of rechecking the stable/mitaka change in nova once it lands12:32
*** ccamacho is now known as ccamacho|lunch12:33
*** dprince has joined #openstack-infra12:35
*** rbuzatu_ has joined #openstack-infra12:39
*** sdake has quit IRC12:39
*** raildo has quit IRC12:40
*** rbuzatu__ has joined #openstack-infra12:40
*** raildo has joined #openstack-infra12:41
*** rbuzat___ has joined #openstack-infra12:42
*** rbuzatu has quit IRC12:42
*** thorst_afk has joined #openstack-infra12:43
*** rbuzatu_ has quit IRC12:43
*** rbuzatu has joined #openstack-infra12:43
*** hashar has joined #openstack-infra12:44
*** rbuzatu_ has joined #openstack-infra12:44
*** rbuzatu__ has quit IRC12:45
*** faizy has quit IRC12:45
*** rbuzatu__ has joined #openstack-infra12:46
*** sdake has joined #openstack-infra12:46
*** rbuzat___ has quit IRC12:47
*** rbuzat___ has joined #openstack-infra12:47
*** rbuzatu has quit IRC12:47
*** rbuzatu_ has quit IRC12:49
*** rbuzatu__ has quit IRC12:50
*** rbuzat___ has quit IRC12:51
*** erlon has joined #openstack-infra12:51
openstackgerritMerged openstack-infra/project-config master: Layout: Make gate-tempest-dsvm-full-devstack-plugin-ceph-trusty non-voting  https://review.openstack.org/43540612:57
*** yamamoto has quit IRC12:58
*** mtanino has joined #openstack-infra12:59
*** sdake_ has joined #openstack-infra12:59
*** lucasagomes is now known as lucas-hungry13:01
*** yamamoto has joined #openstack-infra13:01
*** Hal has quit IRC13:02
*** sdake has quit IRC13:02
*** rbuzatu has joined #openstack-infra13:05
*** yamamoto has quit IRC13:06
*** trown|outtypewww is now known as trown13:08
*** sflanigan has quit IRC13:10
*** jsho has joined #openstack-infra13:12
*** fguillot has joined #openstack-infra13:13
*** kjackal_ has joined #openstack-infra13:14
*** admcleod has joined #openstack-infra13:14
*** admcleod has joined #openstack-infra13:14
*** sdake has joined #openstack-infra13:14
*** mtanino has quit IRC13:15
*** sdake_ has quit IRC13:15
*** EricGonczer_ has joined #openstack-infra13:16
*** yuanying has quit IRC13:18
*** salv-orlando has joined #openstack-infra13:22
*** EricGonc_ has joined #openstack-infra13:24
*** EricGonczer_ has quit IRC13:24
*** nadya has quit IRC13:26
*** shardy_lunch is now known as shardy13:32
*** kornicameister has quit IRC13:32
*** xyang1 has joined #openstack-infra13:33
*** andreas_s has quit IRC13:34
*** Hal has joined #openstack-infra13:35
*** Goneri has joined #openstack-infra13:35
*** links has quit IRC13:36
*** ccamacho|lunch is now known as ccamacho13:37
*** dingyichen has quit IRC13:39
pabelangermudpuppy: yes, we are getting flaky networking from the instances, in same cases we can boot the VMs, test jobs run, but tcp connection break.13:41
*** faizy has joined #openstack-infra13:41
pabelangermudpuppy: in other cases, we don't actually bring a VM online 100%, because we have a DNS check in place, which fails to resolve the outside world.13:41
pabelangermudpuppy: for now, we've disabled osic-cloud1, but you should be able to use our tenant to reproduce the issues13:42
openstackgerritVasyl Saienko proposed openstack-infra/project-config master: Made ironic-multinode job voting for Queens+  https://review.openstack.org/43224613:42
*** jheroux has joined #openstack-infra13:44
*** gordc has joined #openstack-infra13:45
*** baoli has joined #openstack-infra13:46
*** EricGonc_ has quit IRC13:47
*** rlandy has joined #openstack-infra13:48
*** jaosorior has quit IRC13:50
*** martinkopec_ has quit IRC13:50
*** esikache1 has quit IRC13:51
*** jaosorior has joined #openstack-infra13:51
*** pradk has joined #openstack-infra13:53
*** mdrabe has joined #openstack-infra13:54
*** jaosorior has quit IRC13:56
*** jaosorior has joined #openstack-infra13:56
*** thorst_afk is now known as thorst_13:56
*** dprince has quit IRC13:57
*** abregman has joined #openstack-infra13:59
*** dprince has joined #openstack-infra14:00
mordredAJaeger: the manage-projects fixes havent' landed: https://review.openstack.org/#/q/status:open+branch:master+topic:fix-manage-projects14:01
mordredAJaeger: we should wrangle some folks to review them14:01
*** rbuzatu has quit IRC14:01
*** gouthamr has joined #openstack-infra14:01
*** rbuzatu has joined #openstack-infra14:02
*** andymaier has joined #openstack-infra14:02
*** jbernard has quit IRC14:03
sigmavirusHi all, so I'm still trying to get docker to work well for craton. My latest attempt is to create an `/etc/defaults/docker` file with some options for the docker daemon service, but I'm getting permission denied even (https://review.openstack.org/#/c/435038/12/tools/test-setup.sh, https://www.wired.com/2017/02/guide-getting-past-customs-digital-privacy-intact/) I assume our Jenkins user isn't allowed to do that?14:03
*** rbuzatu_ has joined #openstack-infra14:03
*** jbernard has joined #openstack-infra14:04
*** rbuzatu__ has joined #openstack-infra14:04
mordredsigmavirus: sudo echo "DOCKER_OPTS=\"--group=$(id -gn)\"" >> /etc/default/docker doesn't carry the sudo across the >>14:05
sigmavirusTIL14:05
*** abregman has quit IRC14:05
sigmavirusthanks  mordred !14:05
mordredsigmavirus: lemme find a trick to link you to14:05
fungiyeah, the indirection is processed by the calling shell before sudo is ever invoked14:05
sigmavirusThat makes sense14:05
*** yamamoto has joined #openstack-infra14:05
pabelangerecho foo | sudo tee /tmp/blah14:05
fungisomething like sudo sh -c 'echo foo >> bar'14:05
sigmaviruspabelanger: ah nice14:06
*** rbuzat___ has joined #openstack-infra14:06
fungiyep, several ways to skin that cat14:06
*** krtaylor has quit IRC14:06
mordredsigmavirus: https://github.com/openstack/diskimage-builder/blob/63a27bd0c452ff5e97e11e1e79251402c978c12d/elements/dpkg/pre-install.d/00-disable-apt-recommends#L10-L1314:06
sigmavirusheh14:06
*** eharney has joined #openstack-infra14:06
mordredsigmavirus: if you sudo dd of=/etc/default/docker << the sudo will apply to the dd command and all will be good14:06
*** rbuzatu has quit IRC14:06
sigmavirusnice14:07
sigmavirusI like pabelanger's better though =P14:07
*** rbuzatu has joined #openstack-infra14:07
fungiseveral ways indeed14:07
*** HenryG_ is now known as HenryG14:07
openstackgerritMerged openstack-infra/jeepyb master: Continue to optimize manage-projects for resiliency  https://review.openstack.org/43333314:07
*** rbuzatu_ has quit IRC14:08
mordredpabelanger always wins14:08
AJaegermordred: ah - so, what needs to be done for the repo? Land the changes? Or manual github admin?14:08
*** rbuzatu_ has joined #openstack-infra14:08
mordredAJaeger: land the change14:08
AJaegerpabelanger: could you review https://review.openstack.org/#/q/status:open+branch:master+topic:fix-manage-projects , please?14:08
pabelangerAJaeger: yes, doing so now14:08
mordred\o/14:09
*** jamesdenton has joined #openstack-infra14:09
openstackgerritJakub Libosvar proposed openstack-infra/project-config master: neutron: Move linuxbridge Neutron scenario to -nv check queue  https://review.openstack.org/43538814:09
*** rbuzatu__ has quit IRC14:09
*** rbuzatu__ has joined #openstack-infra14:09
*** rbuzat___ has quit IRC14:10
AJaegerfungi, could you review the manage-project changes by mordred as well, please?14:10
fungii have been. approved one a few minutes ago, looking over the track-upstream split change now but it's large14:11
*** yamamoto has quit IRC14:11
*** rbuzat___ has joined #openstack-infra14:11
AJaegerthanks, fungi!14:11
AJaegerand thanks, pabelanger14:11
*** kgiusti has joined #openstack-infra14:11
*** rbuzatu has quit IRC14:12
*** rbuza____ has joined #openstack-infra14:12
*** rbuzatu_ has quit IRC14:12
*** mriedem has quit IRC14:13
*** rbuzatu has joined #openstack-infra14:13
*** jamesdenton has quit IRC14:13
*** rbuzatu__ has quit IRC14:14
*** rbuzatu_ has joined #openstack-infra14:15
*** chlong has joined #openstack-infra14:15
*** rbuzat___ has quit IRC14:15
*** rbuzatu__ has joined #openstack-infra14:16
*** rbuza____ has quit IRC14:16
*** rbuzat___ has joined #openstack-infra14:17
*** rbuzatu has quit IRC14:17
fungimordred: see pabelanger's fresh comments on 433650 (i'll approving it if you want to address that cleanup in another change)14:18
fungier, i'll approve it14:18
AJaegerthree review suggestions from myself for project-config, please: https://review.openstack.org/435260 (removes now unused db jobs - still some left for trove) and https://review.openstack.org/435299 to remove some liberty branch conditions; also https://review.openstack.org/431501 to fix openstack-ansible jobs14:18
mordredfungi: I look now14:18
fungilow blood-coffee level still14:18
*** lucas-hungry is now known as lucasagomes14:18
mordredyah. srrsly14:18
*** rbuzatu has joined #openstack-infra14:18
mordredpabelanger: good catches14:19
*** cshastri has quit IRC14:19
*** rbuzatu_ has quit IRC14:19
*** rbuzatu_ has joined #openstack-infra14:20
*** rbuzatu__ has quit IRC14:20
*** rbuzatu__ has joined #openstack-infra14:21
*** rbuzat___ has quit IRC14:21
*** jaosorior has quit IRC14:22
*** rbuzat___ has joined #openstack-infra14:22
*** kgiusti has left #openstack-infra14:23
*** rbuzatu has quit IRC14:23
openstackgerritMonty Taylor proposed openstack-infra/jeepyb master: Deduplicate some more code  https://review.openstack.org/43546914:23
*** rbuzatu has joined #openstack-infra14:24
mordredpabelanger, fungi: ^^ there ya go14:24
*** rbuzatu_ has quit IRC14:24
*** rustlebee is now known as russellb14:24
fungiteh coolnez14:24
*** russellb has quit IRC14:24
*** russellb has joined #openstack-infra14:24
*** rbuzatu_ has joined #openstack-infra14:25
mordredfungi, pabelanger: if we don't watch out, this will actually work14:25
*** rbuzatu__ has quit IRC14:25
pabelanger+214:26
*** rbuzatu__ has joined #openstack-infra14:26
*** rbuzat___ has quit IRC14:26
*** nadya has joined #openstack-infra14:27
*** rbuzat___ has joined #openstack-infra14:27
*** mtanino has joined #openstack-infra14:28
*** alexchadin has joined #openstack-infra14:28
*** burgerk has joined #openstack-infra14:28
*** rbuzatu has quit IRC14:28
*** ihrachys has joined #openstack-infra14:29
*** rbuzatu_ has quit IRC14:29
*** aviau has quit IRC14:30
*** amotoki has joined #openstack-infra14:30
*** aviau has joined #openstack-infra14:30
openstackgerritSergey Kulanov proposed openstack-infra/jeepyb master: Allow to customize values in .gitreview file  https://review.openstack.org/41013714:30
*** jaosorior has joined #openstack-infra14:30
*** rbuzatu__ has quit IRC14:30
*** jamesdenton has joined #openstack-infra14:30
*** kgiusti has joined #openstack-infra14:31
*** jsho has quit IRC14:31
*** rbuzat___ has quit IRC14:32
openstackgerritMerged openstack-infra/jeepyb master: Split track-upstream from manage-projects  https://review.openstack.org/43365014:32
*** dansmith is now known as superdan14:32
openstackgerritMerged openstack-infra/jeepyb master: Correct a boolean inversion for github updating  https://review.openstack.org/43367714:32
*** pradk has quit IRC14:34
*** tnovacik has joined #openstack-infra14:34
*** jsho has joined #openstack-infra14:35
mordredpabelanger: while we're jeepybing - that patch from sergey looks pretty sane14:36
*** dprince has quit IRC14:37
*** amotoki has quit IRC14:38
*** dprince has joined #openstack-infra14:38
*** esikache1 has joined #openstack-infra14:40
openstackgerritMerged openstack-infra/jeepyb master: Skip retry loop on first check for group  https://review.openstack.org/41306414:42
*** agarner_away has quit IRC14:42
openstackgerritMerged openstack-infra/jeepyb master: Move make_local_copy to utils  https://review.openstack.org/43384814:42
openstackgerritMerged openstack-infra/jeepyb master: Only expire open reviews  https://review.openstack.org/26725014:43
*** trown is now known as trown|brb14:43
*** aleph1 has joined #openstack-infra14:43
fungimordred: i left you a comment on 43366314:43
funginot sure what you actually meant there14:44
*** yanyanhu has joined #openstack-infra14:44
*** zz_dimtruck is now known as dimtruck14:44
openstackgerritMonty Taylor proposed openstack-infra/jeepyb master: Deduplicate some more code  https://review.openstack.org/43546914:44
*** _Cyclone_ has joined #openstack-infra14:44
openstackgerritMerged openstack-infra/jeepyb master: Make protocol configurable  https://review.openstack.org/34613914:44
*** spzala has joined #openstack-infra14:45
openstackgerritPaul Belanger proposed openstack-infra/nodepool feature/zuulv3: Disable test_image_upload_fail  https://review.openstack.org/43547714:45
*** esikache1 has quit IRC14:45
mordredfungi: oh - I may just kinda suck14:45
mordredfungi: I was thinking "let's run this every hour" - so that's just * right?14:46
fungiyeah14:46
mordred*headdesk*14:46
openstackgerritMerged openstack-infra/jeepyb master: Allow to customize values in .gitreview file  https://review.openstack.org/41013714:46
fungicool, i just couldn't tell if that's what you actually wanted (in which case what you had there would have likely accomplished the same thing, just looked strange)14:46
*** trown|brb is now known as trown14:46
openstackgerritMonty Taylor proposed openstack-infra/system-config master: Run track-upstream in a cronjob  https://review.openstack.org/43366314:47
mordredfungi: fixeded14:47
fungiawesome, thanks!14:48
openstackgerritDavid Shrewsbury proposed openstack-infra/nodepool feature/zuulv3: Fix possible race with node request handling  https://review.openstack.org/43512814:48
*** tonytan4ever has joined #openstack-infra14:53
*** rossella_s has quit IRC14:53
*** dtantsur is now known as dtantsur|afk14:53
*** mriedem has joined #openstack-infra14:53
*** esikache1 has joined #openstack-infra14:53
*** tlian has joined #openstack-infra14:53
dmsimardmmedvede: trying to understand your last comment on https://review.openstack.org/#/c/416584/ -- is that caused by the python2 versus python3 thing ?14:53
*** rossella_s has joined #openstack-infra14:54
*** ihrachys has quit IRC14:54
*** ihrachys has joined #openstack-infra14:57
*** esikache1 has quit IRC14:58
*** jamesdenton has quit IRC14:59
*** Julien-zte has joined #openstack-infra15:00
*** jamesdenton has joined #openstack-infra15:01
*** hongbin has joined #openstack-infra15:02
*** pradk has joined #openstack-infra15:03
mmedvededmsimard: py2 vs py3 is at least part of the problem there. It does try to use non-existing path for ara15:03
openstackgerritPaul Belanger proposed openstack-infra/nodepool feature/zuulv3: Disable NodeCleanupWorker() for test_image_upload_fail  https://review.openstack.org/43548115:03
*** psachin has quit IRC15:04
dmsimardmmedvede: ack -- btw I started working slowly on py315:04
dmsimardMost unit tests pass but there's still a couple failures15:04
dmsimardI fixed the low hanging fruits but there's some non-obvious things (i.e, byte vs unicode)15:05
openstackgerritPaul Belanger proposed openstack-infra/nodepool feature/zuulv3: Disable NodeCleanupWorker() for test_image_upload_fail  https://review.openstack.org/43548115:05
*** vsaienk0 has joined #openstack-infra15:06
vsaienk0dtantsur, AJaeger please review https://review.openstack.org/#/c/432246/15:06
mmedvededmsimard: thanks! I need to test it at some point, wrong path in d-g makes it a bit harder15:09
*** rbrndt has joined #openstack-infra15:09
*** rderose has joined #openstack-infra15:11
*** jsho has quit IRC15:12
*** tnovacik has quit IRC15:14
*** psachin has joined #openstack-infra15:15
*** saibarspeis has quit IRC15:16
*** esikache1 has joined #openstack-infra15:16
openstackgerritMerged openstack-infra/shade master: Convert use of .register_uri to .register_uris  https://review.openstack.org/43451615:17
*** esikache1 has quit IRC15:21
openstackgerritMerged openstack-infra/shade master: Convert test_object to use .register_uris  https://review.openstack.org/43453215:21
*** faizy has quit IRC15:22
*** armax has joined #openstack-infra15:23
*** salv-orlando has quit IRC15:23
*** ihrachys_ has joined #openstack-infra15:24
*** ihrachys has quit IRC15:25
openstackgerritMerged openstack-infra/nodepool feature/zuulv3: Disable test_image_upload_fail  https://review.openstack.org/43547715:28
*** rbuzatu has joined #openstack-infra15:28
*** ihrachys_ has quit IRC15:28
fungiooh, updated faq from my broadband isp says they anticipate rolling out dual-stack native service to most residential customers early this year!15:28
fungimaybe not much longer and i can drop my tunnel15:29
openstackgerritMerged openstack-infra/nodepool feature/zuulv3: Fix possible race with node request handling  https://review.openstack.org/43512815:29
fungithough i'll probably give up reverse dns delegation and use of an entire /4815:29
*** bnemec is now known as beekneemech15:30
*** EricGonczer_ has joined #openstack-infra15:31
*** ihrachys has joined #openstack-infra15:31
dmsimardfungi: they have this thing here.. ipv6 rapid deployment ? https://en.wikipedia.org/wiki/IPv6_rapid_deployment15:32
dmsimardI just had to toggle something on my router and bam, I had it. Magic.15:32
*** faizy has joined #openstack-infra15:32
fungidmsimard: right, 6rd is just a tunnel, and way worse than the gre tunnel broker i was using15:33
*** esikache1 has joined #openstack-infra15:33
dmsimardIt's at the ISP layer, though, right ? I didn't have to set up anything on my end. So I guess it's as good as the ISP makes it15:33
dmsimardI haven't noticed any issues with it so far as I can tell15:34
dmsimardtelnet on the zuul jobs work just fine :P15:34
*** marst has quit IRC15:34
*** marst has joined #openstack-infra15:34
fungiit's a tunnel through the provider's network to their border, with all the mtu reduction, fragmentation and overloaded spofs that entails15:34
*** nadya has quit IRC15:34
*** Douhet has quit IRC15:35
fungii was trying to use my provider's 6rd gateway for a while, but it was often completely offline leaving me wit broken connectivity15:35
dmsimardright, like I said, it'll be as good as the ISP makes it :)15:35
*** Douhet has joined #openstack-infra15:35
*** ihrachys has quit IRC15:36
fungipoint being, basically no better than configuring your router to connect to a reasonably well-connected and low-latency tunnel broker endpoint15:37
fungiand potentially worse if your isp doesn't know what they're doing (which mine apparently doesn't/didn't)15:38
*** marst has quit IRC15:39
jeblairclarkb, fungi, pabelanger: anyone know of a host called logstash-worker-test.openstack.org?15:41
*** alexchadin has quit IRC15:41
fungithat one's unfamiliar to me15:41
jeblairit's trying to send mail and does not have rdns configured so some of it is bouncing.15:43
openstackgerritGabriele Cerami proposed openstack-infra/tripleo-ci master: [WIP] Quickstart transition: replace toci_gate_test.sh  https://review.openstack.org/43102915:43
fungidoesn't seem to have fdns configured either for that matter15:44
*** marst has joined #openstack-infra15:46
pabelangerjeblair: not me, and okay for it to be deleted15:47
*** esikache1 has quit IRC15:47
*** armax has quit IRC15:48
openstackgerritPaul Belanger proposed openstack-infra/nodepool feature/zuulv3: Disable CleanupWorker thread for test_image_upload_fail  https://review.openstack.org/43548115:48
*** krtaylor has joined #openstack-infra15:49
openstackgerritPaul Belanger proposed openstack-infra/nodepool feature/zuulv3: Disable CleanupWorker thread for test_image_upload_fail  https://review.openstack.org/43548115:50
fungii'm going to guess it's something clarkb was testing, maybe vintage circa our trusty upgrade sprint?15:52
*** john-davidge has quit IRC15:53
*** ijw has joined #openstack-infra15:53
*** john-davidge has joined #openstack-infra15:53
pabelanger/dev/mapper/main-logs          12T   12T  305G  98% /srv/static/logs15:55
pabelangerthat is better :)15:55
*** psachin has quit IRC15:56
fungioh, heads up, i'll be missing in action for a good chunk of today (annual tax prep appointment followed by the usual drowning of sorrows at a bar)15:56
*** Hal has quit IRC15:56
pabelangerbottoms up15:56
clarkbya I dont recall what logstash-worker-test was for if I booted it. Feel free to delete15:57
fungiprobably vanishing from irc around 17:30 or so15:57
fungi(utc)15:57
*** xarses_ has joined #openstack-infra15:57
clarkbthe trustyification is probably good guess15:57
jeblairinfra-root: i'm going to delete logstash-worker-test soon unless someone objects15:58
mordredjeblair: please do15:58
*** Goneri has quit IRC15:58
mordredfungi: I'm impressed that your find is still running15:58
fungimordred: s/impressed/disappointed/ here15:59
*** andreww has joined #openstack-infra15:59
*** rbuzatu_ has joined #openstack-infra15:59
*** yamahata has joined #openstack-infra16:00
*** tqtran has joined #openstack-infra16:00
*** spzala has quit IRC16:00
*** rbuzatu__ has joined #openstack-infra16:01
*** baoli has quit IRC16:01
fungimordred: and we're at 2% free on the volume now16:01
*** xarses_ has quit IRC16:01
fungii was really hoping to see it get further by now16:01
*** baoli has joined #openstack-infra16:01
*** xarses_ has joined #openstack-infra16:01
*** xarses_ has quit IRC16:02
*** rbuzat___ has joined #openstack-infra16:02
*** xarses_ has joined #openstack-infra16:02
*** rbuzatu has quit IRC16:03
*** yanyanhu has quit IRC16:03
*** rbuzatu has joined #openstack-infra16:03
*** andreww has quit IRC16:03
*** knangia has joined #openstack-infra16:03
*** rbuzatu_ has quit IRC16:03
*** sdake has quit IRC16:04
*** tqtran has quit IRC16:04
*** rbuzatu_ has joined #openstack-infra16:05
openstackgerritAndrea Frittoli proposed openstack/gertty master: Fix gertty py3.x incompatibility  https://review.openstack.org/43550616:05
*** rbuzatu__ has quit IRC16:05
*** sdake has joined #openstack-infra16:06
*** rbuzatu__ has joined #openstack-infra16:06
*** rbuzat___ has quit IRC16:06
*** rderose has quit IRC16:07
*** rbuzat___ has joined #openstack-infra16:07
*** rbuzatu has quit IRC16:07
*** rbuzatu has joined #openstack-infra16:09
*** rbuzatu_ has quit IRC16:09
*** rbuzatu_ has joined #openstack-infra16:10
*** rderose has joined #openstack-infra16:10
*** wolverineav has joined #openstack-infra16:10
*** tlian2 has joined #openstack-infra16:10
*** markus_z has quit IRC16:10
*** kzaitsev_mb has quit IRC16:10
jeblairfungi: if we've increased our storage rate recently, it would stand to reason the reduction won't be linear, and we'll end up with more than 75% full.16:10
*** rbuzatu__ has quit IRC16:10
*** kzaitsev_mb has joined #openstack-infra16:11
fungiagreed, and i expect that, but 98% full is pretty disappointing if that's where we end up :/16:11
jeblairya16:11
*** rbuzat___ has quit IRC16:12
*** tlian has quit IRC16:12
*** rbuzatu has quit IRC16:13
mordredwe should be able to do a second pass to remove those etc dirs16:13
*** rbuzatu_ has quit IRC16:14
*** rossella_s has quit IRC16:14
*** tlian2 has quit IRC16:15
*** ihrachys has joined #openstack-infra16:16
pabelangerwas thinking about it last night, make we configure zuul to prefix pipeline info into log path? Then we can have different retention policy per pipeline.  We could be more aggressive with check (since it is possible to recheck often) and relax for gate16:20
pabelangerI know we do it today, but more it up more to the top level16:20
*** annegentle has joined #openstack-infra16:20
*** pcaruana has quit IRC16:21
*** esikache1 has joined #openstack-infra16:21
*** Apoorva has joined #openstack-infra16:23
*** salv-orlando has joined #openstack-infra16:24
fungiwe could pretty easily adjust our find patterns to do it with the current schema too16:25
fungiwhat would actually make it easier to expire would be if we sharded by date/time16:26
fungithen we wouldn't need to scan the entire tree and could just drop entire subtrees as their expiration date was reached16:27
pabelangergood idea16:28
fungiwonder if we could shard afs volumes by date, and then just drop/recreate like a ring buffer?16:28
*** wolverineav has quit IRC16:28
*** salv-orlando has quit IRC16:29
fungitalked about doing something similar with the subunit2sql tables16:29
jeblairthat would be super easy; we'd lose the directory traversal for a single change though... worth thinking about whether that's valuable16:29
*** jaosorior has quit IRC16:30
fungiyeah, we'd need to maintain a separate index via some automation if we wanted to simulate that16:30
jeblairwe've gone down the separate index path; if we want to do that, we should seriously consider finishing up the swift work :)16:30
*** esikache1 has quit IRC16:30
pabelangerya, I think this idea actually16:30
pabelangers/think/like16:31
fungiright, that dawned on me as i was saying it too ;)16:31
*** aarefiev has quit IRC16:32
AJaegerpabelanger, mordred,fungi, could you put these on your review queue if possible, please? https://review.openstack.org/435260 (removes now unused db jobs - still some left for trove) and https://review.openstack.org/435299 to remove some liberty branch conditions; also https://review.openstack.org/431501 to fix openstack-ansible jobs16:32
*** rcernin has quit IRC16:35
*** andymaier has quit IRC16:35
*** kzaitsev_mb has quit IRC16:37
*** nadya has joined #openstack-infra16:39
*** vponomaryov has left #openstack-infra16:39
openstackgerritJames E. Blair proposed openstack-infra/zuul feature/zuulv3: Add support for roles in zuul  https://review.openstack.org/43517916:40
*** ccamacho has quit IRC16:40
*** tesseract has quit IRC16:41
*** jamesdenton has quit IRC16:42
*** pramodrj07 has quit IRC16:46
*** MasterOfBugs has joined #openstack-infra16:46
*** vsaienk0 has quit IRC16:46
*** kjackal_ has quit IRC16:50
openstackgerritJames E. Blair proposed openstack-infra/zuul feature/zuulv3: Add support for roles in zuul  https://review.openstack.org/43517916:53
*** faizy has quit IRC16:54
*** beslemon has joined #openstack-infra16:56
*** baoli has quit IRC16:58
*** ccamacho has joined #openstack-infra16:59
*** rbrndt has quit IRC17:00
*** spzala has joined #openstack-infra17:00
*** mhickey has quit IRC17:02
*** ZZelle_ has joined #openstack-infra17:04
*** ijw has quit IRC17:05
*** EricGonczer_ has quit IRC17:05
*** jordanP has quit IRC17:07
*** mahendratech has joined #openstack-infra17:07
*** EricGonczer_ has joined #openstack-infra17:08
*** mahendratech has left #openstack-infra17:09
*** e0ne_ has joined #openstack-infra17:09
*** kjackal_ has joined #openstack-infra17:10
*** e0ne_ has quit IRC17:11
*** rbuzatu has joined #openstack-infra17:11
*** ihrachys has quit IRC17:11
*** baoli has joined #openstack-infra17:11
*** rkukura_ has joined #openstack-infra17:12
*** e0ne has quit IRC17:12
*** saibarspeis has joined #openstack-infra17:13
fungiAJaeger: 427895 is waiting on stable/newton to be fixed for trove i guess?17:14
*** ihrachys has joined #openstack-infra17:14
*** rkukura has quit IRC17:15
*** rkukura_ is now known as rkukura17:15
*** kjackal_ has quit IRC17:17
openstackgerritMerged openstack-infra/project-config master: Remove now unused db jobs  https://review.openstack.org/43526017:19
fungiodyssey4me: i left some comments on 431501... not a -1 but want to make sure you're aware that other projects are not running stable/newton jobs on trusty so you may end up needing to drop those jobs before that branch is eol17:20
mudpuppypabelanger, fungi: I tracked down one of the issues to gating for the osic cluster. It looks like ipv6 router was working, but the ipv4 router had issues with the default gateway. That is why api.rubygems.com failed. I believe we resolved that this morning. Can we fire it up?17:20
*** mriedem1 has joined #openstack-infra17:20
fungimudpuppy: sure thing. that also 'splains the dns errors since the resolver is set to an ipv4 addy17:20
pabelangeryup17:21
*** mriedem has quit IRC17:21
pabelangerlet me get a revert up17:21
fungithought there already was one17:21
pabelangerkk17:21
mordred\o/17:21
pabelangerhttps://review.openstack.org/#/c/43527917:21
pabelanger+217:21
odyssey4mefungi will the nodepool nodes be removed entirely? we actually need them to run until newton-eol cc andymccr17:21
pabelangerodyssey4me: which node?17:22
odyssey4meubuntu-trusty17:22
odyssey4mewe use them for newton tests17:22
pabelangershould be around for a while longer17:22
fungiodyssey4me: yea, the idea is that once stable/mitaka reaches eol we can work on decommissioning support for testing on ubuntu-trusty nodes17:23
fungiso it probably won't be immediate, but basically openstack newton as a whole isn't tested on and thus doesn't officially support running on trusty17:23
clarkbthere was a change to stop testing swift against trusty because liberty eol so there may be confusion17:24
clarkbthat change should be abandoned if it hasnt been already17:24
*** mdrabe has quit IRC17:24
fungiclarkb: this time it's in reference to https://review.openstack.org/43527917:24
*** jpich has quit IRC17:24
*** neiljerram has quit IRC17:24
Zara(hm, so yesterday I said I was going to write a story for python-notedb, but since it would be a new project there's not a good place yet. so then I wondered if storyboard should have some area for people to write about projects-to-be, but decided it'd probably be better to just make the project. so may be asking questions about that later (though probably not today))17:25
*** esikache1 has joined #openstack-infra17:25
odyssey4mefungi hmm, ok - I guess we'll have to arrange some sort of external CI for trusty and convert our xenial jobs to voting17:25
fungiodyssey4me: ideally you would get your stable/newton jobs working on ubuntu-xenial nodes soon17:25
odyssey4mefungi they do already, although unfortunately it's a lot slower so we often get timeouts17:25
fungisince if you depend on other openstack components there's no guarantee they won't break you on stable/newton with xenial or later assumptions at some point17:26
odyssey4mefungi we build all the openstack components from the python source, and have had no problems at all17:26
fungiodyssey4me: right... yet anyway17:27
odyssey4mebut yes, our support for trusty on newton is to provide our deployers a transition point17:27
odyssey4meie they get to upgrade to newton/trusty, then upgrade asap to newton/xenial17:27
clarkbsome projects (gnocchi iirc) require newer mysql than available on trusty in newton17:27
*** derekh has quit IRC17:27
clarkbso there are gotchas already17:28
odyssey4meclarkb we use mariadb from the mariadb repositories - we don't rely solely on distro packages17:28
*** annegentle has quit IRC17:28
odyssey4methat's likely why we've not had issues at all17:28
clarkbodyssey4me: yes thats fine just explaining how these things can start breaking17:28
*** esikache1 has quit IRC17:29
*** annegentle has joined #openstack-infra17:29
odyssey4meyeah, we'll figure something out17:30
*** hashar has quit IRC17:30
fungianyway, just wanted to make sure you're aware you may not be able to continue running those jobs on trusty nodes for the full lifespan of the stable/newton branch17:30
fungias a project we consider trusty support to end with mitaka17:31
fungiso we do test upgrading from mitaka to newton on trusty, but once stable/mitaka reaches eol we drop the corresponding grenade jobs similarly17:32
odyssey4meyeah, thanks for the headsup fungi17:32
*** mdrabe has joined #openstack-infra17:32
fungicool, approving in that case17:33
openstackgerritMerged openstack-infra/project-config master: Revert "Revert "Revert "Temporarily disable osic-cloud1"""  https://review.openstack.org/43527917:33
*** dimtruck is now known as zz_dimtruck17:33
fungiclarkb: heh, just realized i linked to the wrong change for what we were discussing. was 43150117:34
*** esikache1 has joined #openstack-infra17:35
AJaegerfungi: yeah ;( trove's newton branch is broken - amrith is currently travelling and will look afterwards into it17:36
fungiokay, i need to get out of here. may be back around 21:30 utc17:36
openstackgerritTim Burke proposed openstack-infra/project-config master: Stop swifts func-post-as-copy job running on stable  https://review.openstack.org/43466417:38
*** ihrachys has quit IRC17:38
*** dprince has quit IRC17:38
clarkbfungi: gl17:39
*** ihrachys has joined #openstack-infra17:39
*** salv-orlando has joined #openstack-infra17:40
*** esikache1 has quit IRC17:40
*** dprince has joined #openstack-infra17:40
*** admcleod has quit IRC17:40
clarkbfungi: https://review.openstack.org/#/c/435129/ is the change I was thinking of. Looks like AJaeger already commented on it17:42
*** rbuzatu_ has joined #openstack-infra17:42
*** Goneri has joined #openstack-infra17:42
openstackgerritsebastian marcet proposed openstack-infra/openstackid-resources master: Updated feedback serializer  https://review.openstack.org/43555017:42
*** eroux has joined #openstack-infra17:43
*** lucasagomes is now known as lucas-afk17:43
*** rbuzatu__ has joined #openstack-infra17:43
openstackgerritMerged openstack-infra/tripleo-ci master: Add converge update to major upgrade test  https://review.openstack.org/43425017:44
*** rbuzat___ has joined #openstack-infra17:45
*** rbuzatu has quit IRC17:45
*** anilvenkata has quit IRC17:45
*** zz_dimtruck is now known as dimtruck17:46
*** rbuzatu_ has quit IRC17:46
*** rbuzatu has joined #openstack-infra17:46
openstackgerritMerged openstack-infra/project-config master: OSA: Close trusty job leaks for Ocata onwards  https://review.openstack.org/43150117:46
AJaegerclarkb: yeah, was surprised to see that ;(17:47
*** rbuzatu_ has joined #openstack-infra17:47
AJaegerclarkb: do you have time for a cleanup change to review, please? https://review.openstack.org/#/c/435299/17:47
*** rbuzatu__ has quit IRC17:48
clarkbAJaeger: gerrit says that cannot merge, but I will review it and see what happens if I approve it17:49
*** rbuzatu__ has joined #openstack-infra17:49
*** rbuzat___ has quit IRC17:49
*** esikache1 has joined #openstack-infra17:49
*** matrohon has quit IRC17:50
AJaegerinteresting - I don't see that in the gui. But let's see and I'll rebase if needed17:50
*** rbuzat___ has joined #openstack-infra17:50
*** rbuzatu has quit IRC17:50
*** reed has quit IRC17:50
*** reed has joined #openstack-infra17:50
*** rbuzatu has joined #openstack-infra17:51
*** rbuzatu_ has quit IRC17:52
*** Julien-zte has quit IRC17:52
clarkbAJaeger: I approved it. The "Cannot Merge" that I see is right under the topic in the top block of things17:53
*** spzala has quit IRC17:53
*** Julien-zte has joined #openstack-infra17:53
*** rbuzatu__ has quit IRC17:53
*** Julien-zte has quit IRC17:53
openstackgerritMerged openstack-infra/openstackid-resources master: Updated feedback serializer  https://review.openstack.org/43555017:53
*** esikache1 has quit IRC17:53
openstackgerritTim Burke proposed openstack-infra/project-config master: Stop swifts func-post-as-copy job running on stable  https://review.openstack.org/43466417:54
*** Julien-zte has joined #openstack-infra17:54
*** tosky has quit IRC17:54
*** Julien-zte has quit IRC17:54
*** Julien-zte has joined #openstack-infra17:54
*** rbuzat___ has quit IRC17:55
*** Julien-zte has quit IRC17:55
*** Julien-zte has joined #openstack-infra17:55
*** Julien-zte has quit IRC17:56
*** Julien-zte has joined #openstack-infra17:56
clarkbmriedem1: looks like your d-g fix got in17:56
*** rbuzatu has quit IRC17:56
*** Julien-zte has quit IRC17:56
AJaegerclarkb: Now I see it as well, ok will rebase...17:57
clarkbAJaeger: well it might merge?17:57
clarkbAJaeger: maybe wait tosee what happens?17:57
clarkbI dunno17:57
*** Julien-zte has joined #openstack-infra17:57
AJaegerok ;)17:57
*** nadya has quit IRC17:57
*** Julien-zte has quit IRC17:57
AJaegerrebases fine locally, so can push if needed ;)17:58
*** yamahata has quit IRC18:01
*** matrohon has joined #openstack-infra18:02
pabelangerfungi: mudpuppy: things already look better in for osic-cloud1: http://grafana.openstack.org/dashboard/db/nodepool-osic18:04
pabelangerthanks for the help!18:04
mudpuppypabelanger: Cool. Everyone should use ipv6! :)18:04
*** ihrachys has quit IRC18:04
pabelanger++18:04
openstackgerritAndreas Jaeger proposed openstack-infra/project-config master: Remove liberty branch conditions from some projects  https://review.openstack.org/43529918:05
AJaegerclarkb: I had to rebase ;( ^18:05
clarkbAJaeger: ok will rereview real quick18:06
AJaegerthanks18:07
*** rbrndt has joined #openstack-infra18:07
*** ijw_ has joined #openstack-infra18:08
AJaegerclarkb, pabelanger : Would be nice to merge my infra-manual change to document test-setup.sh, please: https://review.openstack.org/43278618:09
*** salv-orl_ has joined #openstack-infra18:10
*** ihrachys has joined #openstack-infra18:10
*** esikache1 has joined #openstack-infra18:10
clarkblooking18:10
clarkbI will probably be in and out today as I prep for PTG. Have plans for tomorrow nad early flight sunday which means getting stuff done today18:11
*** sdake_ has joined #openstack-infra18:12
*** salv-orlando has quit IRC18:12
*** sdake has quit IRC18:14
openstackgerritMerged openstack-infra/infra-manual master: Document tools/test-setup.sh  https://review.openstack.org/43278618:14
*** ijw_ has quit IRC18:15
*** tlian has joined #openstack-infra18:18
AJaegerthanks, clarkb.18:19
*** armax has joined #openstack-infra18:19
*** ociuhandu has quit IRC18:20
*** ijw has joined #openstack-infra18:20
*** annegentle has quit IRC18:21
*** pblaho has quit IRC18:23
*** jamesdenton has joined #openstack-infra18:23
AJaegerclarkb, pabelanger, one more loose end: mordred's changes for jeepyb. Could you help reviewing, please? https://review.openstack.org/#/q/status:open+branch:master+topic:fix-manage-projects - otherwise somebody should fix https://github.com/openstack/vmware-vspc manually18:23
*** trown is now known as trown|lunch18:24
*** esikache1 has quit IRC18:24
*** rbrndt has quit IRC18:27
openstackgerritMerged openstack-infra/project-config master: Remove liberty branch conditions from some projects  https://review.openstack.org/43529918:28
*** mriedem1 is now known as mriedem18:30
mriedemclarkb: yup thanks for pushing that through18:30
*** dfflanders has quit IRC18:31
*** EricGonc_ has joined #openstack-infra18:32
*** EricGonczer_ has quit IRC18:34
*** john-davidge has quit IRC18:35
*** ihrachys has quit IRC18:36
openstackgerritAndreas Jaeger proposed openstack-infra/project-config master: Remove final db setup  https://review.openstack.org/42789518:37
AJaegerclarkb: your python3 devstack-gate change is in - can we merge https://review.openstack.org/417555 now?18:38
*** ihrachys has joined #openstack-infra18:38
*** yamahata has joined #openstack-infra18:39
AJaegerarmax, kevinbenton : Is kevinbenton now also Neutron Infra liason? If yes, please add to https://wiki.openstack.org/wiki/CrossProjectLiaisons18:40
armaxAJaeger: for the time being I am still happy to be in that capacity18:41
armaxif kevinbenton lets me18:41
*** sambetts is now known as sambetts|afk18:42
AJaegerarmax: I'm wondering whether kevinbenton's +1 is a "Neutron Infra liason +1" on https://review.openstack.org/#/c/435388/ - or whether I need to ping you or njohnston_ ;)18:42
AJaegerarmax: happy to have you around ;)18:42
*** MasterOfBugs has quit IRC18:42
*** electrofelix has quit IRC18:43
*** esikache1 has joined #openstack-infra18:43
armaxAJaeger: kevinbenton is the PTL after all18:44
*** beslemon has quit IRC18:46
AJaegerarmax: I know - that's why I ask ;) I'm in pedantic mood but will take his +1 for now ;)18:48
openstackgerritDavid Moreau Simard proposed openstack-infra/puppet-openstackci master: Allow logs(-dev).o.o to serve gzipped svg files  https://review.openstack.org/43556618:48
*** tqtran has joined #openstack-infra18:51
*** rbuzatu has joined #openstack-infra18:52
morganbefore I bail out for the weekend (and other lovely -infra folks), I'm getting a bunch of 502s on gerrit atm (cc pabelanger, clarkb, fungi), and 1-3 minutes to load changesets. [figure letting you know now helps prevent weekend craziness]18:56
*** nunchuck has joined #openstack-infra18:57
clarkbmorgan: seems to be working for me. Is it on a specific change or page?18:57
pabelangerI just got a proxy error18:58
dmsimardI got proxy errors as well just now18:58
johnsomI am also seeing gerrit slowness18:58
morganclarkb: like 4-5 different changesets18:58
*** spzala has joined #openstack-infra18:58
pabelangerchecking GC now18:58
pabelangerOh, ya. I see a spike18:59
clarkbhuh maybe I just got lucky18:59
morganpabelanger: ty! :) it doesn't matter to me too much personally, but if we head it off now, no weekend ick when people are on the way to the PTG :)18:59
pabelangerclarkb: are you okay for restart?19:00
pabelangerchecking pipelines now19:00
pabelangerreleases look clean19:00
clarkbya if its GCing that serms to be our only option19:01
AJaegergo for it, please ;)19:01
pabelanger#status notice Restarting gerrit due to performance problems19:02
openstackstatuspabelanger: sending notice19:02
* morgan goes back under a rock until the next time raising the "oh hey gerrit might need to be restarted" flag :P19:02
*** spzala has quit IRC19:02
-openstackstatus- NOTICE: Restarting gerrit due to performance problems19:03
pabelangerokay, gerrit stopped and started again19:04
*** ihrachys has quit IRC19:04
*** spzala has joined #openstack-infra19:04
openstackstatuspabelanger: finished sending notice19:05
*** hashar has joined #openstack-infra19:05
AJaegerlooks fine again, thanks19:06
openstackgerritAndreas Jaeger proposed openstack-infra/project-config master: Remove mysql setup  https://review.openstack.org/43556919:06
openstackgerritDavid Moreau Simard proposed openstack-infra/project-config master: Add openstack-ansible non-voting jobs to ARA check queue  https://review.openstack.org/43557019:06
*** MasterOfBugs has joined #openstack-infra19:06
*** ihrachys has joined #openstack-infra19:07
*** dharinic is now known as dharinic|lunch19:07
openstackgerritDavid Moreau Simard proposed openstack-infra/project-config master: Add openstack-ansible non-voting jobs to ARA check queue  https://review.openstack.org/43557019:07
*** spzala has quit IRC19:09
*** sdake_ has quit IRC19:10
*** fguillot has quit IRC19:12
*** gyee has joined #openstack-infra19:14
openstackgerritMerged openstack-infra/tripleo-ci master: Remove openstack-heat-templates from bootstrap elements path  https://review.openstack.org/43510219:15
*** spzala has joined #openstack-infra19:15
openstackgerritMerged openstack-infra/tripleo-ci master: Stop mirroring centos images  https://review.openstack.org/43037219:15
*** wolverineav has joined #openstack-infra19:16
*** Rockyg has joined #openstack-infra19:18
*** aimeeu has quit IRC19:18
*** aimeeu has joined #openstack-infra19:19
*** _ody has joined #openstack-infra19:19
*** spzala has quit IRC19:20
*** sbezverk has left #openstack-infra19:21
openstackgerritVasyl Saienko proposed openstack-infra/project-config master: Made ironic-multinode job voting for Pike+  https://review.openstack.org/43224619:24
*** rbuzatu_ has joined #openstack-infra19:25
*** rbrndt has joined #openstack-infra19:25
*** rbuzatu__ has joined #openstack-infra19:26
*** rbuzat___ has joined #openstack-infra19:27
*** rbuzatu has quit IRC19:28
*** rbuzatu_ has quit IRC19:29
*** annegentle has joined #openstack-infra19:30
*** rbuzatu__ has quit IRC19:30
*** rbuzat___ has quit IRC19:31
*** tnovacik has joined #openstack-infra19:32
*** armax has quit IRC19:33
*** ihrachys has quit IRC19:34
*** rbuzatu has joined #openstack-infra19:35
*** john-davidge has joined #openstack-infra19:36
*** ociuhandu has joined #openstack-infra19:37
*** ihrachys has joined #openstack-infra19:39
*** spzala has joined #openstack-infra19:40
*** john-davidge has quit IRC19:41
*** trown|lunch is now known as trown19:43
*** esikachev has joined #openstack-infra19:44
*** spzala has quit IRC19:44
*** claudiub has joined #openstack-infra19:46
*** esikache1 has quit IRC19:48
*** ccamacho has quit IRC19:48
*** ssbarnea has quit IRC19:50
*** rfolco has quit IRC19:51
*** eroux has quit IRC19:55
*** tnovacik has quit IRC19:55
*** annegentle has quit IRC19:55
*** tnovacik has joined #openstack-infra19:56
*** dharinic|lunch is now known as dharinic19:56
*** beslemon has joined #openstack-infra19:57
*** annegentle has joined #openstack-infra19:57
*** ihrachys has quit IRC19:58
*** ihrachys has joined #openstack-infra20:00
*** beslemon has quit IRC20:00
*** beslemon has joined #openstack-infra20:11
*** spzala has joined #openstack-infra20:12
*** intlabs is now known as portdirect20:13
openstackgerritMerged openstack/diskimage-builder master: Fix requirements update  https://review.openstack.org/43277120:14
*** beslemon has quit IRC20:16
*** spzala has quit IRC20:16
*** ldnunes has quit IRC20:18
*** portdirect is now known as portdirect_travl20:18
*** cody-somerville has joined #openstack-infra20:20
*** csomerville has quit IRC20:22
*** e0ne has joined #openstack-infra20:25
*** rfolco has joined #openstack-infra20:26
*** rfolco has quit IRC20:26
*** ihrachys has quit IRC20:26
openstackgerritMerged openstack-infra/project-config master: networking-ovn: Reorganize CI jobs.  https://review.openstack.org/39468420:26
pabelangerdham1: mordred: what do you think about tagging a new release of PBR? Running into some issues with PBR in the fedora 26 mass rebuild today: https://kojipkgs.fedoraproject.org//work/tasks/8116/17798116/build.log It looks to be fixed in https://review.openstack.org/#/c/413795/20:26
pabelangerdham1: sorry20:26
pabelangerdhellmann: ^20:27
*** ihrachys has joined #openstack-infra20:27
*** salv-orl_ has quit IRC20:35
*** kjackal_ has joined #openstack-infra20:35
*** rbuzatu has quit IRC20:37
*** rbuzatu has joined #openstack-infra20:38
*** saibarspeis has quit IRC20:38
*** beslemon has joined #openstack-infra20:38
*** rbuzatu_ has joined #openstack-infra20:39
*** rbuzatu__ has joined #openstack-infra20:40
*** rbuzatu has quit IRC20:42
*** beslemon has quit IRC20:43
*** rbuzatu_ has quit IRC20:44
openstackgerritSachin proposed openstack-infra/project-config master: Introducing Omni  https://review.openstack.org/43336320:46
openstackgerritSachin proposed openstack-infra/project-config master: Introducing Omni Omni is a set of public cloud drivers for OpenStack. A set of drivers for relevant services: Nova, Glance, Cinder and Neutron are provided, which let OpenStack provision workloads on public clouds. Eventually, Omni will be multi-cloud/hyb  https://review.openstack.org/43336320:47
*** raildo has quit IRC20:48
openstackgerritSachin proposed openstack-infra/project-config master: Introducing Omni Omni is a set of public cloud drivers for OpenStack. A set of drivers for relevant services: Nova, G lance, Cinder and Neutron are provided, which let OpenStack provision workloads on public clouds. Eventually, Omni will be multi-cloud/hy  https://review.openstack.org/43336320:50
*** mtanino has quit IRC20:51
*** dprince has quit IRC20:51
openstackgerritSachin proposed openstack-infra/project-config master: Introducing Omni  https://review.openstack.org/43336321:00
*** beslemon has joined #openstack-infra21:06
openstackgerritRicardo Carrillo Cruz proposed openstack-infra/devstack-gate master: Ansibly the network sanity check  https://review.openstack.org/40424321:09
fungipabelanger: i expect pbr releases need to wait for at least thursday just to be safe21:10
*** annegentle has quit IRC21:10
pabelangerfungi: Yup, that is what I confirmed21:10
pabelangerthanks21:10
funginp21:10
openstackgerritGabriele Cerami proposed openstack-infra/tripleo-ci master: [WIP] Quickstart transition: replace toci_gate_test.sh  https://review.openstack.org/43102921:11
*** spzala has joined #openstack-infra21:12
*** kjackal_ has quit IRC21:14
*** armax has joined #openstack-infra21:14
openstackgerritRicardo Carrillo Cruz proposed openstack-infra/devstack-gate master: Ansibly the network sanity check  https://review.openstack.org/40424321:14
*** chem has quit IRC21:15
openstackgerritRicardo Carrillo Cruz proposed openstack-infra/devstack-gate master: Ansibly the network sanity check  https://review.openstack.org/40424321:21
*** kzaitsev_mb has joined #openstack-infra21:21
*** dimtruck is now known as zz_dimtruck21:22
*** kjackal_ has joined #openstack-infra21:27
*** tnovacik has quit IRC21:28
openstackgerritRicardo Carrillo Cruz proposed openstack-infra/devstack-gate master: Ansibly the network sanity check  https://review.openstack.org/40424321:29
*** kgiusti has left #openstack-infra21:30
*** tonytan4ever has quit IRC21:34
*** Jeffrey4l_ has joined #openstack-infra21:34
*** burgerk has quit IRC21:34
*** Jeffrey4l has quit IRC21:34
*** annegentle has joined #openstack-infra21:37
*** john-davidge has joined #openstack-infra21:37
*** beslemon has quit IRC21:39
*** beslemon has joined #openstack-infra21:40
*** john-davidge has quit IRC21:41
*** annegentle has quit IRC21:41
*** e0ne has quit IRC21:43
*** ihrachys has quit IRC21:44
*** ihrachys has joined #openstack-infra21:45
*** wolverineav has quit IRC21:48
*** tonytan4ever has joined #openstack-infra21:48
*** wolverineav has joined #openstack-infra21:49
*** gordc has quit IRC21:49
*** thorst_ has quit IRC21:53
*** jamesdenton has quit IRC21:54
*** amotoki has joined #openstack-infra21:57
*** amotoki has quit IRC21:58
*** hashar has quit IRC21:59
*** amotoki has joined #openstack-infra21:59
*** amotoki has quit IRC21:59
*** wolverineav has quit IRC22:00
openstackgerritMerged openstack-infra/zuul feature/zuulv3: Correct playbook requirement logic  https://review.openstack.org/43456422:00
openstackgerritMerged openstack-infra/zuul feature/zuulv3: Reload tenant configuration on change merged  https://review.openstack.org/42643322:01
*** trown is now known as trown|outtypewww22:01
openstackgerritMerged openstack-infra/zuul feature/zuulv3: Add support for roles in zuul  https://review.openstack.org/43517922:01
*** wolverineav has joined #openstack-infra22:03
*** wolverineav has quit IRC22:07
openstackgerritRicardo Carrillo Cruz proposed openstack-infra/devstack-gate master: Ansibly the network sanity check  https://review.openstack.org/40424322:09
rcarrillocruzfungi: speechless about the infracloud rail-less servers...22:11
*** gouthamr has quit IRC22:11
*** ihrachys has quit IRC22:12
clarkbrcarrillocruz: I fondly remember a server named "shelf"22:13
rcarrillocruzheh22:13
*** rbrndt has quit IRC22:15
*** ihrachys has joined #openstack-infra22:15
fungircarrillocruz: yeah, insane22:16
*** dave-mccowan has quit IRC22:17
fungiat one point i worked for a cheapskate hoster and we had a "no more than three servers per railset" policy22:17
fungieven that was insane22:17
*** wolverineav has joined #openstack-infra22:17
fungi"oh, we need to take these other two systems offline so we can swap out ram in the one with the rails"22:17
rcarrillocruzwhich reinforces my feeling we did a titanic effort on keeping gozer up and running, given the top-notch DC we had there...22:17
*** kjackal_ has quit IRC22:18
*** rlandy has quit IRC22:18
rcarrillocruzclarkb: ^ that is the last of the ansibly setup host thing22:18
rcarrillocruzi think it is functional22:18
fungijust reinforces my thoughts about how hp(e) should never have managed data centers22:18
rcarrillocruzat least passed in one dsvm node i ssh'd in22:18
rcarrillocruzfungi: 10x ++22:18
*** rbuzatu__ has quit IRC22:19
*** rbuzatu has joined #openstack-infra22:20
fungigranted, vendors over-charge for rails. back in the day i remember the railmount kits for mainstream rack-format servers increased the overall cost by something like 10%22:20
rcarrillocruzoh yeah22:20
clarkbfungi: hp is the vendor though22:20
fungiright22:20
rcarrillocruzi remember doing research for my home rackmount cases22:20
fungiwhich is what makes this extra-funny22:20
rcarrillocruzcomparing cheapo chinese kits vs HP stuff22:21
rcarrillocruzHP was ludicrously expensive22:21
fungialso anything 3u or thicker you _could_ reasonably flange-mount (unless it was a disk array anyway)22:22
fungibut i'm guessing these are all 2u models22:22
*** wolverineav has quit IRC22:23
*** jheroux has quit IRC22:24
*** salv-orlando has joined #openstack-infra22:26
*** beslemon has quit IRC22:26
openstackgerritIhar Hrachyshka proposed openstack-infra/devstack-gate master: Enable memory_tracker in gate  https://review.openstack.org/43447422:29
*** ihrachys has quit IRC22:30
*** sdake has joined #openstack-infra22:30
*** baoli has quit IRC22:33
*** gouthamr has joined #openstack-infra22:34
*** beslemon has joined #openstack-infra22:34
*** rbrndt has joined #openstack-infra22:36
*** aspire9984 has joined #openstack-infra22:36
*** aspire9984 has left #openstack-infra22:37
*** john-davidge has joined #openstack-infra22:37
*** beslemon has quit IRC22:38
*** rhallisey has quit IRC22:40
*** john-davidge has quit IRC22:42
*** wolverineav has joined #openstack-infra22:47
*** matrohon has quit IRC22:48
*** zz_dimtruck is now known as dimtruck22:49
*** jd__ has quit IRC22:59
*** xyang1 has quit IRC23:02
mordredhp also charged itself retail rates23:02
mordredso the folks in teh DC there would be buying all of that kit at full consumer price out of their budgets23:02
mordred*boggles*23:02
*** beslemon has joined #openstack-infra23:02
fungibizarre23:03
*** tonytan4ever has quit IRC23:05
*** mdrabe has quit IRC23:10
*** Hunner_ is now known as Hunner23:12
*** beslemon has quit IRC23:15
*** beslemon has joined #openstack-infra23:15
*** beslemon has quit IRC23:17
*** ijw has quit IRC23:18
*** spzala has quit IRC23:19
*** armax has quit IRC23:22
openstackgerritGabriele Cerami proposed openstack-infra/tripleo-ci master: [WIP] Quickstart transition: replace toci_gate_test.sh  https://review.openstack.org/43102923:23
*** esikachev has quit IRC23:24
*** mriedem has quit IRC23:26
*** thorst has joined #openstack-infra23:27
openstackgerritEmilien Macchi proposed openstack-infra/tripleo-ci master: remove test-environments/multinode_major_upgrade.yaml  https://review.openstack.org/42637523:29
openstackgerritEmilien Macchi proposed openstack-infra/tripleo-ci master: remove test-environments/multinode_major_upgrade.yaml  https://review.openstack.org/42637523:29
*** ZZelle_ has quit IRC23:30
*** claudiub|2 has joined #openstack-infra23:30
*** MasterOfBugs has quit IRC23:32
*** thorst has quit IRC23:32
*** claudiub has quit IRC23:32
*** jkilpatr has quit IRC23:34
*** wolverineav has quit IRC23:37
*** wolverineav has joined #openstack-infra23:41
*** marst has quit IRC23:41
*** esikachev has joined #openstack-infra23:45
*** chlong has quit IRC23:46
*** wolverineav has quit IRC23:50
*** esikachev has quit IRC23:50
*** esikachev has joined #openstack-infra23:52
*** wolverineav has joined #openstack-infra23:55
*** marst has joined #openstack-infra23:55
*** wolverineav has quit IRC23:59

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!