Friday, 2019-08-09

*** rlandy|rover has quit IRC00:09
*** rfolco has joined #oooq00:29
*** rf0lc0 has joined #oooq00:35
*** rfolco has quit IRC00:36
*** rfolco has joined #oooq01:16
*** rf0lc0 has quit IRC01:18
*** apetrich has quit IRC02:08
*** rfolco has quit IRC02:21
*** ade_lee__ has joined #oooq02:36
*** altlogbot_0 has quit IRC02:44
*** altlogbot_2 has joined #oooq02:44
*** ade_lee__ has quit IRC02:47
*** ade_lee__ has joined #oooq02:48
*** dsneddon has quit IRC03:03
*** dsneddon has joined #oooq03:11
*** dsneddon has quit IRC03:16
*** surpatil has joined #oooq03:17
*** redrobot has quit IRC03:22
*** dsneddon has joined #oooq03:57
*** dsneddon has quit IRC04:05
*** dsneddon has joined #oooq04:34
*** raukadah is now known as chkumar|ruck04:38
*** surpatil has quit IRC04:53
*** surpatil has joined #oooq04:54
*** ratailor has joined #oooq04:56
*** ykarel has joined #oooq05:10
*** udesale has joined #oooq05:33
*** jfrancoa has joined #oooq05:39
*** marios has joined #oooq05:39
*** panda has quit IRC05:47
*** panda has joined #oooq05:48
*** dsneddon has quit IRC05:54
*** jaosorior has joined #oooq06:12
*** dsneddon has joined #oooq06:22
*** dtantsur|afk is now known as dtantsur06:28
*** apetrich has joined #oooq06:37
*** dsneddon has quit IRC06:42
*** dsneddon has joined #oooq07:00
*** tesseract has joined #oooq07:04
*** kopecmartin|off is now known as kopecmartin07:04
*** jfrancoa has quit IRC07:07
*** amoralej|off is now known as amoralej07:07
*** jfrancoa has joined #oooq07:23
*** tosky has joined #oooq07:23
*** bogdando has joined #oooq07:25
zbrmorning!07:46
marioszbr: yes, can confirm07:48
marioschecks out in the eastern med.07:48
zbrchkumar|ruck: marios : see https://review.opendev.org/#/c/673859/ -- had link to taiga in last comment.07:48
marioszbr: is it copy paste of07:49
marios*=* 10:02:20 *=*=*= "Add placeholder tox-molecule job"  https://review.opendev.org/#/c/674571/3 I8374e1648ef44796b73068ca8061be3ef39d113a07:49
*** ratailor has quit IRC07:49
marioshm not quite but ack zbr07:49
*** ykarel is now known as ykarel|lunch07:50
marioschkumar|ruck: o/ why did you rebase https://review.opendev.org/#/c/674228/ was there a conflict?07:51
chkumar|ruckmarios: scenario 10 nv patch merged recently, so I rebased it, there was no merge conflict, but wanted to make it passing,07:53
marioschkumar|ruck: you can just recheck?07:53
chkumar|ruckrecheck should have also solved the problem07:53
marioschkumar|ruck: ack thanks for looking07:53
zbrarxcruz: ping08:09
*** bhagyashris has joined #oooq08:10
zbrchkumar|ruck: where was config_template module hosted?08:20
*** ratailor has joined #oooq08:22
*** jaosorior has quit IRC08:29
*** dsneddon has quit IRC08:37
chkumar|ruckzbr: https://opendev.org/openstack/ansible-config_template consumed in tripleo ansible and os_tempest08:44
*** chem has joined #oooq08:57
*** jaosorior has joined #oooq09:01
*** derekh has joined #oooq09:05
zbrchkumar|ruck: found it few minutes later.09:10
*** dsneddon has joined #oooq09:10
arxcruzzbr:pong09:14
arxcruzzbr: pong09:14
arxcruzzbr: sorry, wen't to the office today09:15
*** ykarel|lunch is now known as ykarel09:15
zbrarxcruz: nothing yet. i am redoing the os_tempest role testing09:15
zbri went into a problem where ansible fails to find the role because its checkout/repo folder name does not match the role name.09:17
arxcruzi'm against change the role name09:18
zbrthis breaks ansible well enough because role lookup relies on folder name.09:18
zbrbut I can find workarounds. the real role name seems to be "os_tempest" but somehow nobody even bodered to record that inside meta.yml09:19
zbrlong story short: adding tests uncovers lots of previous issues. is like looking under the carpet... ;)09:19
zbrarxcruz: lucky for us, symlinks can save us again.09:22
sshnaidm|offmarios, https://review.opendev.org/#/c/667219/09:24
mariossshnaidm|off: k i guess there's always revert if necessary :D09:26
mariossshnaidm|off: if you aren't going to have your day off i'll have it please?09:27
sshnaidm|offmarios, of course)09:27
*** tesseract has quit IRC09:28
sshnaidm|offmarios, I'm off, just wanna look ci exploding :)09:28
mariossshnaidm|off: i am sure chkumar|ruck is forever grateful for your service09:28
*** tesseract has joined #oooq09:28
*** tesseract has joined #oooq09:29
*** bhagyashris has quit IRC10:03
*** ykarel is now known as ykarel|afk10:03
*** ykarel|afk is now known as ykarel10:38
*** jaosorior has quit IRC10:38
*** dsneddon has quit IRC10:40
*** d0ugal has quit IRC10:54
*** jaosorior has joined #oooq10:55
*** d0ugal has joined #oooq11:04
*** rfolco has joined #oooq11:11
*** dsneddon has joined #oooq11:15
*** ajo has left #oooq11:22
*** akahat has joined #oooq11:32
*** rlandy has joined #oooq11:59
*** rlandy is now known as rlandy|rover11:59
rlandy|roverweshay: hi - the 'virtblade chassis from BOS Lab 4 C-09' - that's the old hardware, right?12:03
rlandy|roverchkumar|ruck: hey12:04
rlandy|roverwhat's the damage today?12:04
weshayhey.. few min12:05
chkumar|ruckrlandy|rover: not much, we are in good shape12:06
chkumar|ruckrlandy|rover: neutron tempest broke queens promotion12:06
rlandy|roverchkumar|ruck: wow - gate failures are down12:06
chkumar|ruckrlandy|rover: rhel8 still stuck12:06
rlandy|roverchkumar|ruck: a new failure from neutron - over and above the one on the CIX board?12:06
chkumar|ruckrlandy|rover: yes it is fixed now12:07
arxcruzmarios: https://review.rdoproject.org/r/#/c/21787/ i think we won't need the ansible_python_interpreter, sshnaidm|off had a patch to switch quickstart to ansible 2.8 that will fix the python_ansible_interpreter12:07
rlandy|roverchkumar|ruck: both - failures?12:07
rlandy|roverchkumar|ruck: this is looking much prettier http://rhos-release.virt.bos.redhat.com:3030/rhosp today12:07
rlandy|roverstein finally promoted ... small party12:07
mariosarxcruz: thx commented there12:08
rlandy|roverchkumar|ruck: let's touch base re: rhel-8 when you have a chance12:08
chkumar|ruckrlandy|rover: sure12:08
rlandy|roveralso ovb is looking very instable  - will check into that12:08
chkumar|ruckweshay: rlandy|rover: https://review.opendev.org/#/c/675563/ container build fix12:08
rlandy|roverok - this is the one we spoke about yesterday12:09
chkumar|ruckarxcruz: kopecmartin feel free to add the agenda here https://etherpad.openstack.org/p/tripleo-tempest-squad-meeting12:09
chkumar|ruckrlandy|rover: yes12:09
rlandy|roverwhat about the concerns you had on rebase failures?12:09
chkumar|ruckrlandy|rover: yes that is the problem, I need to push those patch to kolla itself12:10
chkumar|ruckI can take care of tempest fix12:10
rlandy|roverchkumar|ruck: well it's broken right now, so this can't be worse12:11
rfolcochkumar|ruck, tripleo-ci-rhel-8-standalone-rdo is the check job for rhel8 ? still failing, https://review.rdoproject.org/zuul/builds?job_name=tripleo-ci-rhel-8-standalone-rdo12:11
rlandy|roverchkumar|ruck: let's try it - I'll try merge t once zuul votes12:11
chkumar|ruckrfolco: yes that is a know thing, we are waiting for the promotion12:12
*** jaosorior has quit IRC12:12
chkumar|ruckit depends on https://review.opendev.org/#/c/675282/12:12
rfolcochkumar|ruck, what is the upstream replacement for f28-standalone?12:13
rfolcochkumar|ruck, or rhel8 is rdo only12:13
rlandy|roverchkumar|ruck: ugh - error in image upload for rhel-8 - will fix it12:13
chkumar|ruckrfolco: it should be rdo on rhel-8 third party ci jobs12:14
rfolcook thx12:14
arxcruzchkumar|ruck: i don't feel free, it's a terrible world... full of violence...12:15
weshaychkumar|ruck rlandy|rover check http://status.openstack.org/elastic-recheck/ the #2 issue12:15
rlandy|roverweshay: we're in the big leagues now12:16
rlandy|roverthe number 2 most disaster job12:17
*** dsneddon has quit IRC12:18
rlandy|roverchkumar|ruck: https://review.rdoproject.org/r/21789 Remove calls to tripleo-ci-periodic-base/buildimage12:26
rlandy|roversshnaidm|off: panda: ^^ fyi12:27
weshaychkumar|ruck is the rhel8 container build job running check on https://review.opendev.org/#/c/675563/ in 3rd party?12:30
weshaychkumar|ruck rlandy|rover fyi.. node failures in rdo 3rd party afaict12:30
rlandy|roverweshay: ? all rdo is 3rd party - not understanding12:31
weshayrlandy|rover yes.. I'm just being very specific w/ my words12:32
chkumar|ruckweshay: nope no container build job run against it12:33
weshaychkumar|ruck ok.. that's crazy12:33
weshaychkumar|ruck please add one12:33
weshaythat triggers on roles/build-containers12:33
chkumar|ruckrlandy|rover: https://logs.rdoproject.org/64/675564/1/openstack-check/tripleo-build-containers-rhel-8-master/571c02f/job-output.txt.gz#_2019-08-09_08_26_13_28729212:34
chkumar|ruckweshay: adding12:34
weshaychkumar|ruck are the interns added to todays call?12:37
rlandy|roverchkumar|ruck: looking better12:37
chkumar|ruckweshay: yes12:37
*** dsneddon has joined #oooq12:41
chkumar|ruckweshay: rlandy|rover https://review.rdoproject.org/r/#/c/21790/12:41
rlandy|roverchkumar|ruck: voting?12:41
chkumar|ruckrlandy|rover: false12:43
chkumar|ruckdone12:43
rlandy|roverchkumar|ruck: thanks because this jobs traditionally has voting: true https://github.com/rdo-infra/rdo-jobs/blob/master/zuul.d/build-containers.yaml#L1512:44
chkumar|ruckmarios: needs +w on this https://review.opendev.org/#/c/675282/12:45
chkumar|ruckthanks!12:45
*** dsneddon has quit IRC12:45
*** akahat has quit IRC12:47
*** amoralej is now known as amoralej|lunch12:50
chkumar|ruckmarios: thanks :-)12:56
*** udesale has quit IRC12:56
weshaychkumar|ruck rlandy|rover hey.. the cockpit Promotion Overview may need updating12:57
*** udesale has joined #oooq12:57
mariosnp chkumar|ruck12:57
rlandy|roverweshay: for rhel8?12:58
weshayrlandy|rover no no.. it's all red, but should not be12:58
weshayand maybe adding rhel 812:58
chkumar|ruckmarios: https://review.rdoproject.org/r/#/c/21787/ can run it in check jobs here itself so that we can some idea what is going wrong in the job?12:58
rlandy|roverha - k - looking at it12:58
weshaywe can certainly remove f2812:58
chkumar|ruckmarios: adding the 4 jobs in zuul.d/projects.yaml will do the job12:59
*** Guest1857 has joined #oooq12:59
rlandy|roverack13:00
marioschkumar|ruck: we know whats wrong i was trying to use anchors but it fails13:00
marioschkumar|ruck: i commented there13:00
arxcruzChanServ:?13:01
arxcruzchkumar|ruck: ?13:01
chkumar|ruckweshay: meeting time13:01
*** panda has quit IRC13:14
*** panda has joined #oooq13:15
*** ykarel has quit IRC13:29
rlandy|roverwow- did we ever have ruck-rover-alert bot?13:33
rlandy|roveroh - tripleo-ci13:33
*** amoralej|lunch is now known as amoralej13:39
*** aakarsh has quit IRC13:47
rlandy|roverweshay: do you know who controls the alert bot on #tripleo-ci?13:47
rlandy|roveroops13:47
rlandy|rovertripleo13:47
rlandy|roveractually that's #tripleo-ci13:48
*** vinaykns has joined #oooq13:52
rlandy|rovermarios: hi - you were digging around grafana last week ...13:54
mariosrlandy|rover: o/13:55
rlandy|rovermarios: I ma looking for how the alert notifications get populated in the database ... https://github.com/rdo-infra/ci-config/blob/master/ci-scripts/infra-setup/roles/rrcockpit/files/grafana/export-grafana.py#L6413:55
chkumar|ruckrlandy|rover: I have added all the notes on etherpad13:56
chkumar|ruckrlandy|rover: time to logout13:57
chkumar|rucksee ya on monday13:57
*** chkumar|ruck is now known as raukadah13:57
mariosrlandy|rover: was not really looking at that stuff to be honest13:57
rlandy|roverchkumar|ruck: ack - have a good weekend13:57
rlandy|rovermarios: k13:57
mariosrlandy|rover: started poking at getting promotion info into the build object for the jobs but not successful :D13:57
rlandy|roverit's a scary place to start venturing into13:58
rlandy|roverzero doc13:58
rlandy|roverzero design docs13:58
rlandy|rovercode search and hope13:58
*** surpatil has quit IRC13:58
*** aakarsh has joined #oooq14:21
*** ykarel has joined #oooq14:23
*** dsneddon has joined #oooq14:42
*** dsneddon has quit IRC14:46
*** ratailor has quit IRC14:47
zbrwho should I ping to get ssh access to a ppc64 hosts? i want o improve my multi-arch skills14:52
rfolcomjturek, you know ? ^15:01
rfolcozbr, what I know is this http://openpower.ic.unicamp.br/minicloud/15:02
rfolcoyou request a openpower vm (manually)15:03
rlandy|roverweshay: have  a few minutes to talk about rrcockpit changes?15:05
rlandy|rovercan be later this afternoon if that works better15:05
weshayrlandy|rover I have about 20min15:06
zbrrfolco: i completed the form.thanks. i was hoping we would have something like that internally.15:06
rlandy|roverweshay: I need about 515:06
rfolcozbr, talk to tony breeds. He might know.15:06
rlandy|roverbj?15:07
weshayaye15:07
rlandy|roverjoining15:07
*** vinaykns has quit IRC15:07
zbri was considering ordering a rasberrypi as even aarch would be ok.15:07
rfolcozbr, I am trying to push to my namespace hello-world for ppc and x86... but with docker-ce 19, it does not allow me to pull ppc container image.... not sure how to force15:09
rfolcodocker pull --disable-content-trust ppc64le/hello-world15:09
rfolcoUsing default tag: latest15:09
rfolcolatest: Pulling from ppc64le/hello-world15:09
rfolcono matching manifest for linux/amd64 in the manifest list entries15:09
rfolcozbr, do you know ?15:09
*** ykarel is now known as ykarel|away15:09
zbrthat is why I need a shell on a non-x86 machine, so I can see if arch is hardcoded.15:09
*** marios has quit IRC15:09
zbri would not be surprised to discover that operations with foo arch must run on foo arch.15:10
zbrbut with only one arch, I cannot guess.15:10
*** vinaykns has joined #oooq15:12
*** jfrancoa has quit IRC15:14
*** Vorrtex has joined #oooq15:17
*** panda has quit IRC15:19
*** panda has joined #oooq15:20
rfolcoweshay, 1x1?15:31
weshayrfolco give me a few on w/ rlandy|rover15:31
rfolcoweshay, sure ping when free15:31
weshayrfolco ok join15:31
weshayready15:31
rlandy|roverweshay: sshnaidm|off: raukadah: https://review.rdoproject.org/r/21792 Update old promoter alert links15:35
zbrwho can tell me few things about zuul build env? does zuul dumps the inventory to a file so I can access from inside a tox job? or do I need to add some extra pre-run playbook for doing that dump?15:54
rlandy|roverzbr: zuul dumps the inventory to zuul-info/inventory.yaml15:55
rlandy|roverno need to add anything extra15:55
zbrrelative to ~ or cwd?15:56
rlandy|roverzbr: see example https://logs.opendev.org/56/675356/1/gate/openstack-tox-pep8/756391d/zuul-info/inventory.yaml15:56
zbrso mainly if I can load that file, i am under zuul. thanks.15:56
rlandy|roveralways the same dir structure15:56
zbrrlandy|rover++15:56
weshayrlandy|rover do you remember where the patch sagi had up that changed the job_id and job_name criteria that broke stein?16:00
weshayI don't see it in ci-config16:00
*** udesale has quit IRC16:02
raukadahweshay: rlandy|rover how we make code changes in rrcockpit? I have compose up and everything is running there16:05
raukadahdo I need to enter into container and make changes there?16:05
rlandy|roverweshay: yes - it's in review.rdoproject.org/config16:06
rlandy|rovergetting16:06
rlandy|roverweshay: https://github.com/rdo-infra/review.rdoproject.org-config/commit/3d35f4a5e4017d73ca56c0229fb66d59ed8135db16:07
rlandy|roverraukadah: you can make changes directly in gerrit16:08
rlandy|roveronly need a compose to test16:08
rlandy|roveror add to the dashboard16:08
raukadahrlandy|rover: thanks, I will make the changes16:11
*** bogdando has quit IRC16:11
*** Guest1857 is now known as redrobot16:17
*** sabedevops has joined #oooq16:23
rlandy|rovertripleo-rhel-8-buildimage-overcloud-fullSUCCESS in 21m 47s (non-voting)16:28
rlandy|rovertripleo-rhel-8-buildimage-ironic-python-agentSUCCESS in 17m 08s (non-voting)16:28
rlandy|rover^^ working16:28
rlandy|roveryay16:28
*** kopecmartin is now known as kopecmartin|off16:29
*** surpatil has joined #oooq16:31
raukadahzbr: rfolco panda if still around, need some more eyes on this https://review.opendev.org/#/c/675563/5/roles/build-containers/tasks/main.yaml@7716:33
raukadahit appears to be complicated for a single case16:33
raukadahthanks!16:34
*** dsneddon has joined #oooq16:42
*** irclogbot_1 has quit IRC16:47
*** dsneddon has quit IRC16:47
*** irclogbot_1 has joined #oooq16:47
*** ykarel|away has quit IRC16:49
*** irclogbot_1 has quit IRC16:49
*** irclogbot_2 has joined #oooq16:49
*** dtantsur is now known as dtantsur|afk16:50
*** amoralej is now known as amoralej|off16:51
rlandy|roverhmmm ... maybe I need to restart the service on the promoter?16:56
rlandy|roverstill alerting16:56
rlandy|roverweshay; raukadah: promoter alerts are still showing16:59
rlandy|rovergoing to restart telegraf service16:59
weshayk17:00
rlandy|roveron promoter server17:00
*** tesseract has quit IRC17:00
rlandy|rover2019-08-09T17:02:05Z E! [outputs.influxdb]: when writing to [http://192.168.100.240:8086]: Post http://192.168.100.240:8086/write?db=telegraf: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)17:02
rlandy|rover2019-08-09T17:02:05Z E! Error writing to output [influxdb]: could not write any address17:02
rlandy|roverPING 192.168.100.240 (192.168.100.240) 56(84) bytes of data.17:03
rlandy|rover^C17:03
rlandy|rover--- 192.168.100.240 ping statistics ---17:03
rlandy|rover21 packets transmitted, 0 received, 100% packet loss, time 199917:03
rlandy|roverno show17:03
rlandy|roversever is not pingable17:05
*** sabedevops has quit IRC17:09
*** sabedevops has joined #oooq17:10
*** amoralej|off is now known as amoralej17:12
*** amoralej is now known as amoralej|off17:12
rlandy|roverError running /usr/bin/pgrep: exit status 117:13
rlandy|rovernow sever is pingable17:13
*** dsneddon has joined #oooq17:13
*** sabedevops has quit IRC17:32
rlandy|roverweshay: ugh ^^ any wisdom on the current failures?17:33
rlandy|roveradded default sec group17:33
rlandy|roverrrcockpit is pingable now from promoter17:33
rlandy|rover! Error in plugin [inputs.procstat]: E! Error: procstat getting process, exe: [] pidfile: [] pattern: [^python.*dlrnapi_promoter.py .*Fedora-28.*stein.*] user: [] Error running /usr/bin/pgrep: exit status 117:33
openstackrlandy|rover: Error: "inputs.procstat" is not a valid command.17:33
weshayrlandy|rover looks like we missed something whle removing f2817:34
weshaynot sure tbh.. I need more context17:34
rlandy|roverweshay: I am on the promoter server17:34
rlandy|roverrestarted telegraf to pick up change in conf to new server17:35
rlandy|roverserver was not available17:35
rlandy|roveradded default sec group to make it available17:35
rlandy|rovernow this error17:35
rlandy|roverthe /etc patterns are still old17:36
rlandy|roverthat it is complaining about17:36
rlandy|rovernot sure what process it's looking for17:41
rlandy|rover^ looking ofr that17:43
rlandy|rover "url":"http://influxdb:8086",17:45
*** surpatil has quit IRC17:50
weshayrlandy|rover that's the container addr17:54
weshayrlandy|rover want to chat?17:54
rlandy|roverweshay: yeah17:54
*** sabedevops has joined #oooq17:55
rlandy|roverweshay: bj?17:55
weshayaye17:56
*** vinaykns has quit IRC17:57
*** jtomasek has quit IRC18:21
weshayrlandy|rover back19:36
rlandy|roverweshay; still commenting on the doc :)19:37
rlandy|rovergive me 5 more mins19:37
rlandy|roverand I think the promoter reporting may be working19:37
rlandy|roverwill explain19:37
rlandy|roverok  - I'm done19:38
rlandy|roverready to chat when you are19:38
rlandy|roverweshay: ^^19:39
weshayah nice19:40
weshayok.. blue?19:40
rlandy|roverjoining19:40
*** chem has quit IRC20:03
*** Vorrtex has quit IRC20:06
rlandy|roverCOMMIT_HASH=cc26d47faedf3643237cfb494d001d1add50267c20:15
rlandy|roverimage upload is working again :)20:15
rlandy|roverraukadah: ^^20:15
*** fmount has quit IRC20:33
*** fmount has joined #oooq20:36
*** dsneddon has quit IRC20:42
*** derekh has quit IRC20:44
rlandy|roverweshay: this training https://github.com/red-hat-people-team/open-decision-framework/blob/master/ODF-community.pdf?20:45
*** derekh has joined #oooq20:46
*** dsneddon has joined #oooq21:13
*** rlandy|rover has quit IRC21:31
*** ankit08 has joined #oooq21:43
*** dsneddon has quit IRC21:47
*** dsneddon has joined #oooq21:50
*** ankit08 has quit IRC22:05
*** dsneddon has quit IRC22:10
*** panda has quit IRC22:30
*** panda has joined #oooq22:32
*** sabedevops has quit IRC22:33
rfolcozbr, also found this for ppc access... not sure this works22:41
rfolcohttps://osuosl.org/services/powerdev/request_hosting/22:41
*** aakarsh has quit IRC22:45
*** tosky has quit IRC23:32
*** weshay is now known as weshay_pto23:37

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!