| *** smarcet has quit IRC | 00:13 | |
| *** smarcet has joined #openstack-infra | 00:28 | |
| *** zhurong has quit IRC | 00:34 | |
| *** xenos76 has quit IRC | 00:43 | |
| *** smarcet has quit IRC | 00:51 | |
| *** smarcet has joined #openstack-infra | 00:53 | |
| *** zhurong has joined #openstack-infra | 01:15 | |
| *** gregoryo has joined #openstack-infra | 01:28 | |
| *** spsurya has joined #openstack-infra | 01:29 | |
| *** eandersson has quit IRC | 01:36 | |
| *** larainema has joined #openstack-infra | 01:38 | |
| *** eandersson has joined #openstack-infra | 01:39 | |
| *** tkajinam has quit IRC | 01:45 | |
| *** tkajinam has joined #openstack-infra | 01:47 | |
| *** jhesketh has joined #openstack-infra | 02:29 | |
| *** smarcet has quit IRC | 03:22 | |
| *** smarcet has joined #openstack-infra | 03:27 | |
| *** smarcet has quit IRC | 03:42 | |
| *** jamesmcarthur has joined #openstack-infra | 03:48 | |
| *** jamesmcarthur has quit IRC | 04:06 | |
| *** jamesmcarthur has joined #openstack-infra | 04:09 | |
| *** jamesmcarthur has quit IRC | 04:14 | |
| *** jamesmcarthur has joined #openstack-infra | 04:23 | |
| *** ricolin has joined #openstack-infra | 04:47 | |
| *** ykarel has joined #openstack-infra | 04:52 | |
| *** ykarel is now known as ykarel|away | 04:53 | |
| *** jamesmcarthur has quit IRC | 04:56 | |
| *** jtomasek has joined #openstack-infra | 05:02 | |
| *** dychen has joined #openstack-infra | 05:03 | |
| *** dychen has quit IRC | 05:03 | |
| *** dychen has joined #openstack-infra | 05:24 | |
| *** jamesmcarthur has joined #openstack-infra | 05:25 | |
| *** kjackal has joined #openstack-infra | 05:27 | |
| *** jamesmcarthur has quit IRC | 05:33 | |
| *** dychen has quit IRC | 05:43 | |
| *** dchen has quit IRC | 05:43 | |
| *** psachin has joined #openstack-infra | 06:23 | |
| *** jamesmcarthur has joined #openstack-infra | 06:29 | |
| *** jamesmcarthur has quit IRC | 06:34 | |
| *** jbadiapa has joined #openstack-infra | 06:37 | |
| *** pgaxatte has joined #openstack-infra | 06:38 | |
| *** kopecmartin has joined #openstack-infra | 06:53 | |
| *** tesseract has joined #openstack-infra | 06:56 | |
| *** gfidente has joined #openstack-infra | 06:56 | |
| *** slaweq has joined #openstack-infra | 06:59 | |
| *** ginopc has joined #openstack-infra | 07:01 | |
| *** apetrich has joined #openstack-infra | 07:02 | |
| *** trident has quit IRC | 07:04 | |
| *** rcernin has quit IRC | 07:08 | |
| *** trident has joined #openstack-infra | 07:13 | |
| *** dchen has joined #openstack-infra | 07:17 | |
| *** rcernin has joined #openstack-infra | 07:24 | |
| *** kjackal has quit IRC | 07:27 | |
| *** kjackal has joined #openstack-infra | 07:30 | |
| *** jamesmcarthur has joined #openstack-infra | 07:30 | |
| *** ccamacho has joined #openstack-infra | 07:34 | |
| *** tosky_ has joined #openstack-infra | 07:34 | |
| *** jamesmcarthur has quit IRC | 07:35 | |
| *** ccamacho has quit IRC | 07:44 | |
| *** ccamacho has joined #openstack-infra | 07:44 | |
| *** jpena|off is now known as jpena | 07:45 | |
| *** rcernin has quit IRC | 07:48 | |
| *** pcaruana has joined #openstack-infra | 07:49 | |
| *** lucasagomes has joined #openstack-infra | 07:49 | |
| *** ralonsoh has joined #openstack-infra | 07:51 | |
| *** dchen has quit IRC | 07:52 | |
| *** dchen has joined #openstack-infra | 07:53 | |
| *** pkopec has joined #openstack-infra | 08:08 | |
| *** xenos76 has joined #openstack-infra | 08:17 | |
| *** arxcruz is now known as arxcruz_pto | 08:19 | |
| *** tkajinam has quit IRC | 08:21 | |
| *** panda|rover|off is now known as panda|rover | 08:23 | |
| *** dchen has quit IRC | 08:25 | |
| *** dtantsur|afk is now known as dtantsur | 08:26 | |
| *** jamesmcarthur has joined #openstack-infra | 08:31 | |
| *** tosky_ is now known as tosky | 08:32 | |
| *** jaosorior has joined #openstack-infra | 08:34 | |
| *** jamesmcarthur has quit IRC | 08:36 | |
| *** derekh has joined #openstack-infra | 08:39 | |
| *** gregoryo has quit IRC | 08:41 | |
| *** ociuhandu has joined #openstack-infra | 08:47 | |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/devstack-gate master: Prepare for stable branching in devstack-gate https://review.opendev.org/679610 | 09:07 |
|---|---|---|
| *** jistr has quit IRC | 09:09 | |
| *** jistr has joined #openstack-infra | 09:10 | |
| *** apetrich has quit IRC | 09:13 | |
| *** yamamoto has quit IRC | 09:16 | |
| *** yamamoto has joined #openstack-infra | 09:17 | |
| *** yamamoto has quit IRC | 09:17 | |
| *** xek has joined #openstack-infra | 09:17 | |
| *** yamamoto has joined #openstack-infra | 09:17 | |
| *** yamamoto has quit IRC | 09:22 | |
| *** e0ne has joined #openstack-infra | 09:29 | |
| *** dayou has quit IRC | 09:30 | |
| *** dayou has joined #openstack-infra | 09:31 | |
| *** jamesmcarthur has joined #openstack-infra | 09:33 | |
| *** jamesmcarthur has quit IRC | 09:44 | |
| openstackgerrit | Sorin Sbarnea proposed zuul/zuul-jobs master: add-build-sshkey: add centos/rhel-8 support https://review.opendev.org/674092 | 09:48 |
| *** Garyx has joined #openstack-infra | 10:08 | |
| *** ociuhandu has quit IRC | 10:19 | |
| *** bexelbie_ has joined #openstack-infra | 10:22 | |
| *** bexelbie_ has quit IRC | 10:23 | |
| *** jaosorior has quit IRC | 10:24 | |
| *** rascasoft has joined #openstack-infra | 10:24 | |
| *** jamesmcarthur has joined #openstack-infra | 10:29 | |
| *** jamesmcarthur has quit IRC | 10:34 | |
| *** xek has quit IRC | 10:35 | |
| openstackgerrit | Sorin Sbarnea proposed zuul/zuul-jobs master: add-build-sshkey: add centos/rhel-8 support https://review.opendev.org/674092 | 10:41 |
| *** ociuhandu has joined #openstack-infra | 10:44 | |
| *** dchen has joined #openstack-infra | 11:02 | |
| *** lucasagomes has quit IRC | 11:04 | |
| *** apetrich has joined #openstack-infra | 11:09 | |
| *** jamesmcarthur has joined #openstack-infra | 11:14 | |
| *** Lucas_Gray has joined #openstack-infra | 11:17 | |
| *** ykarel|away has quit IRC | 11:22 | |
| *** ociuhandu has quit IRC | 11:22 | |
| *** jaosorior has joined #openstack-infra | 11:22 | |
| *** ociuhandu has joined #openstack-infra | 11:22 | |
| *** jamesmcarthur has quit IRC | 11:24 | |
| *** smarcet has joined #openstack-infra | 11:25 | |
| *** jamesmcarthur has joined #openstack-infra | 11:25 | |
| *** smarcet has quit IRC | 11:26 | |
| *** smarcet has joined #openstack-infra | 11:27 | |
| zbr | fungi: tristanC clarkb ^ ready | 11:30 |
| *** jamesmcarthur has quit IRC | 11:35 | |
| *** smarcet has quit IRC | 11:35 | |
| *** jpena is now known as jpena|lunch | 11:39 | |
| *** ociuhandu has quit IRC | 11:43 | |
| *** yamamoto has joined #openstack-infra | 11:44 | |
| *** yamamoto has quit IRC | 11:51 | |
| *** rfolco has joined #openstack-infra | 12:04 | |
| *** jamesmcarthur has joined #openstack-infra | 12:09 | |
| *** FlorianFa has joined #openstack-infra | 12:10 | |
| *** panda|rover is now known as panda|rover|eat | 12:13 | |
| *** jamesmcarthur has quit IRC | 12:13 | |
| *** yamamoto has joined #openstack-infra | 12:14 | |
| *** ociuhandu has joined #openstack-infra | 12:14 | |
| *** ociuhandu has quit IRC | 12:19 | |
| *** yamamoto has quit IRC | 12:20 | |
| *** ociuhandu has joined #openstack-infra | 12:20 | |
| *** kjackal_v2 has joined #openstack-infra | 12:22 | |
| *** priteau has joined #openstack-infra | 12:23 | |
| *** kjackal has quit IRC | 12:24 | |
| *** xek has joined #openstack-infra | 12:28 | |
| *** yamamoto has joined #openstack-infra | 12:32 | |
| *** jpena|lunch is now known as jpena | 12:32 | |
| *** xek has quit IRC | 12:35 | |
| *** yamamoto has quit IRC | 12:36 | |
| *** panda|rover|eat is now known as panda|rover | 12:36 | |
| *** smarcet has joined #openstack-infra | 12:40 | |
| *** jamesmcarthur has joined #openstack-infra | 12:45 | |
| *** tkajinam has joined #openstack-infra | 12:47 | |
| *** bhavikdbavishi has joined #openstack-infra | 12:53 | |
| *** yamamoto has joined #openstack-infra | 12:54 | |
| *** yamamoto has quit IRC | 12:58 | |
| *** tkajinam has quit IRC | 12:58 | |
| *** tkajinam has joined #openstack-infra | 12:59 | |
| *** ociuhandu has quit IRC | 13:06 | |
| *** tkajinam has quit IRC | 13:06 | |
| *** ociuhandu has joined #openstack-infra | 13:10 | |
| *** ociuhandu has quit IRC | 13:10 | |
| *** bhavikdbavishi has quit IRC | 13:10 | |
| *** ociuhandu has joined #openstack-infra | 13:10 | |
| guilhermesp | morning infra team! So we've been noticing some breakage across ohv mirrors for centos deployments https://zuul.opendev.org/t/openstack/build/ac40d7c7c1ea466bb3eeb44f9e6e1194/log/job-output.txt#8366 | 13:13 |
| guilhermesp | are you awere of any issues with those mirrors so far? of how we can contact to check them | 13:14 |
| fungi | guilhermesp: ralonsoh reported it in here yesterday. that's just a proxy to trunk.rdoproject.org which was not responding | 13:15 |
| fungi | please double-check that's been fixed before we assume there's anything wrong with the proxies | 13:15 |
| ralonsoh | fungi, now it is https://trunk.rdoproject.org/centos7-master/current/delorean.repo | 13:15 |
| ralonsoh | but not the proxy | 13:15 |
| ralonsoh | https://openstack.fortnebula.com:13808/v1/AUTH_e8fd161dc34c421a979a9e6421f823e9/logs_99/678599/1/check/networking-ovn-tripleo-ci-centos-7-containers-multinode/d43f1cc/job-output.txt | 13:16 |
| fungi | the error messages there are "operation too slow" so maybe access to it from france is not so great right now? | 13:16 |
| fungi | or maybe trunk.rdoproject.org is up but overloaded and slow to respond? | 13:17 |
| noonedeadpunk | fungi: I'd say that https://trunk-primary.rdoproject.org/centos7/2f/6c/2f6cc5cdffa11d44d4652dee337d12e9f440396b_a47d70e7/repodata/repomd.xml is working | 13:17 |
| noonedeadpunk | But http://mirror.gra1.ovh.openstack.org:8080/rdo/centos7/2f/6c/2f6cc5cdffa11d44d4652dee337d12e9f440396b_a47d70e7/repodata/repomd.xml returns 503 | 13:17 |
| fungi | i can't get to trunk.rdoproject.org though, which is what it looks like we're using | 13:22 |
| fungi | just times out for me, both from the command line on mirror.gra1.ovh and also from my workstation at home in the usa | 13:23 |
| fungi | are you *sure* it's back on line | 13:24 |
| fungi | ? | 13:24 |
| fungi | Connecting to trunk.rdoproject.org (trunk.rdoproject.org)|54.81.116.189|:443... | 13:24 |
| fungi | just sits forever for me | 13:24 |
| fungi | guilhermesp: ralonsoh: noonedeadpunk: ^ | 13:25 |
| fungi | anyway, i need to go run some errands, but will be back in a while | 13:25 |
| noonedeadpunk | fungi: http://paste.openstack.org/show/769486/ | 13:25 |
| ralonsoh | fungi, I can reach https://trunk.rdoproject.org/centos7-master/current/delorean.repo | 13:25 |
| noonedeadpunk | BUT, it has not valid SSl and redirects to https | 13:26 |
| fungi | are you able to get https://trunk.rdoproject.org/centos7/2f/6c/2f6cc5cdffa11d44d4652dee337d12e9f440396b_a47d70e7/repodata/repomd.xml though? | 13:26 |
| guilhermesp | same output as noonedeadpunk here from my desktop in brazil | 13:26 |
| noonedeadpunk | which probqably wasn't a case previously | 13:26 |
| fungi | that's what just spins forever for me | 13:26 |
| fungi | anyway, back in a while | 13:26 |
| *** rfarr has joined #openstack-infra | 13:27 | |
| *** Wryhder has joined #openstack-infra | 13:28 | |
| *** Lucas_Gray has quit IRC | 13:29 | |
| *** Wryhder is now known as Lucas_Gray | 13:29 | |
| *** xek has joined #openstack-infra | 13:42 | |
| *** yamamoto has joined #openstack-infra | 13:44 | |
| *** ykarel has joined #openstack-infra | 13:46 | |
| *** jamesmcarthur has quit IRC | 13:49 | |
| openstackgerrit | Liam Young proposed openstack/project-config master: New charm for cinder integration with Purestorage https://review.opendev.org/679652 | 13:54 |
| *** yamamoto has quit IRC | 13:55 | |
| *** larainema has quit IRC | 13:56 | |
| *** yamamoto has joined #openstack-infra | 13:56 | |
| clarkb | note trunk.rdoproject.org is a vname for trunk-backup.rdoproject.org. This has a completely dofferent IP address than trunk-primary.rdoproject.org | 13:58 |
| clarkb | our proxy uses trunk.rdoproject.org so your tests against -primary are trsting acompletely different host | 13:58 |
| clarkb | *cname | 13:58 |
| *** ricolin_ has joined #openstack-infra | 13:59 | |
| frickler | they seem to do dns balancing, sometimes I get a cname to trunk-backup, sometimes to -primary. the latter seems to work, the former doesn't | 14:00 |
| clarkb | ah | 14:01 |
| clarkb | looks like ssl doesnt work on -primary either | 14:01 |
| clarkb | I dont think our proxy is to blame here | 14:01 |
| *** ricolin has quit IRC | 14:02 | |
| *** rfarr has quit IRC | 14:11 | |
| *** ricolin_ is now known as ricolin | 14:22 | |
| *** yamamoto has quit IRC | 14:24 | |
| *** Lucas_Gray has quit IRC | 14:25 | |
| *** yamamoto has joined #openstack-infra | 14:27 | |
| *** pgaxatte has quit IRC | 14:31 | |
| *** roman_g has joined #openstack-infra | 14:38 | |
| *** ociuhandu has quit IRC | 14:45 | |
| fungi | yep. seems our servers are working as expected | 14:49 |
| fungi | rdo either needs to fix whatever's broken with trunk-backup or take it out of the round-robin for trunk | 14:49 |
| *** ykarel has left #openstack-infra | 14:50 | |
| *** yamamoto has quit IRC | 14:52 | |
| *** xek has quit IRC | 14:52 | |
| *** kopecmartin is now known as kopecmartin|off | 14:54 | |
| *** yamamoto has joined #openstack-infra | 15:02 | |
| *** yamamoto has quit IRC | 15:02 | |
| *** psachin has quit IRC | 15:03 | |
| *** pkopec has quit IRC | 15:04 | |
| *** priteau has quit IRC | 15:09 | |
| *** trident has quit IRC | 15:17 | |
| *** ociuhandu has joined #openstack-infra | 15:18 | |
| *** pkopec has joined #openstack-infra | 15:22 | |
| *** ociuhandu has quit IRC | 15:23 | |
| *** ociuhandu has joined #openstack-infra | 15:23 | |
| *** trident has joined #openstack-infra | 15:24 | |
| *** jamesmcarthur has joined #openstack-infra | 15:25 | |
| noonedeadpunk | fungi: so trunk.rdoproject.org for me roslves to cname trunk-primary.rdoproject.org which works just fine... | 15:29 |
| *** jamesmcarthur has quit IRC | 15:31 | |
| *** efried has left #openstack-infra | 15:36 | |
| *** mgoddard has quit IRC | 15:41 | |
| *** bhavikdbavishi has joined #openstack-infra | 15:43 | |
| *** mgoddard has joined #openstack-infra | 15:43 | |
| *** pkopec has quit IRC | 15:45 | |
| *** dtantsur is now known as dtantsur|afk | 15:45 | |
| *** ociuhandu has quit IRC | 15:49 | |
| fungi | noonedeadpunk: it looks like they're using some sort of glb via dns, because here's how it resolves for me: http://paste.openstack.org/show/769638/ | 15:51 |
| fungi | and some of our mirrors are seeing the same dns resolution | 15:51 |
| *** zbr is now known as zbr|ooo | 15:52 | |
| fungi | PLEASE bring this up with whoever maintains trunk.rdoproject.org | 15:52 |
| fungi | there's nothing WE can do about it | 15:52 |
| noonedeadpunk | oh, I see now. | 15:53 |
| clarkb | fungi: noonedeadpunk but also trunk.rdoproject.org then redirects via http 301 to trunk-primary.rdoproject.org which has a cert for trunk.rdoproject.org and not trunk-primary.rdoproject.org | 15:53 |
| clarkb | I think ^ is what you get when the glb dns record is for trunk-primary | 15:53 |
| clarkb | and you get connection timeouts if told to talk to trunk-backup | 15:53 |
| clarkb | both problems will prevent our proxy from serving those files to you | 15:53 |
| noonedeadpunk | But can't we just temporary set resolver to trunk-primary.rdoproject.org ? Like via hosts or whatever? | 15:53 |
| clarkb | no | 15:54 |
| clarkb | because then ssl will break too | 15:54 |
| clarkb | that won't fix it | 15:54 |
| noonedeadpunk | ah, yep... | 15:54 |
| clarkb | RDO needs to fix the source of truth then we'll be able to fetch from it | 15:54 |
| *** pkopec has joined #openstack-infra | 16:01 | |
| *** smarcet has quit IRC | 16:02 | |
| *** jamesmcarthur has joined #openstack-infra | 16:02 | |
| *** xek has joined #openstack-infra | 16:02 | |
| fungi | also, we're not going to waste our time working around rdo's problems. they need to solve them, or your jobs need to stop depending on it | 16:03 |
| *** jamesmcarthur has quit IRC | 16:06 | |
| *** smarcet has joined #openstack-infra | 16:08 | |
| *** smarcet has quit IRC | 16:09 | |
| *** ralonsoh has quit IRC | 16:10 | |
| *** ralonsoh has joined #openstack-infra | 16:20 | |
| sean-k-mooney | anyone know where i can find an example of how to add a periodic job? | 16:20 |
| sean-k-mooney | i assume i just add it to the perodici pipeline but can i say how often it runs? | 16:20 |
| sean-k-mooney | neutron has https://github.com/openstack/neutron/blob/master/.zuul.yaml#L84-L88 | 16:21 |
| sean-k-mooney | but im not sure if we have more contol then that available? | 16:21 |
| clarkb | sean-k-mooney: the pipeline configuration determines how often it runs | 16:21 |
| sean-k-mooney | ah ok | 16:21 |
| sean-k-mooney | the default should be fine | 16:21 |
| sean-k-mooney | looks like every 6hours yes? https://github.com/openstack/project-config/blob/master/zuul.d/pipelines.yaml#L183-L195 | 16:22 |
| sean-k-mooney | or is that every day at 6am | 16:23 |
| clarkb | once a day at 6am utc | 16:23 |
| sean-k-mooney | ok i can never remeber how cron timeing syntax works | 16:23 |
| sean-k-mooney | and we can view the output in the builds tab in the zuul ui | 16:24 |
| clarkb | yes | 16:24 |
| sean-k-mooney | does the voteing state matter for the periodic pipeline | 16:24 |
| clarkb | no | 16:25 |
| sean-k-mooney | e.g. does it ever make sense for a job to periodic and non voting | 16:25 |
| sean-k-mooney | ok i assumed as much | 16:25 |
| clarkb | periodic pipelines don't run against changes so there is nothing for them to vote on | 16:25 |
| clarkb | instead they run against the specified branch head at that trigger point in time | 16:25 |
| sean-k-mooney | ya they just run againt the head of master or whatever branch the jobs is defiend for right | 16:25 |
| sean-k-mooney | yep | 16:25 |
| sean-k-mooney | cool | 16:25 |
| sean-k-mooney | im building an nfv job that relise on nested virt and a lable form a specific provider. so we want to add it to experimental pipline and run it perodically | 16:26 |
| *** ociuhandu has joined #openstack-infra | 16:27 | |
| sean-k-mooney | just wanted to make sure the periodic pipleline worked teh way i expected | 16:27 |
| fungi | also there's no point in setting voting/non-voting for experimental pipeline jobs | 16:30 |
| sean-k-mooney | ya we dont | 16:30 |
| sean-k-mooney | i origially was writing this job to not need nested virt | 16:31 |
| sean-k-mooney | that version would have been in check and non voting as it would need the nightly build of qemu and run on fedora | 16:31 |
| *** ociuhandu has quit IRC | 16:31 | |
| sean-k-mooney | but im swapping ti to use ubuntu 18.04 + nested vert instead since it will be more stable | 16:31 |
| clarkb | what version of qemu do you need? | 16:32 |
| clarkb | (that is the sort of thing that maybe folks like ubuntu cloud archive might be interested in? I dunno) | 16:33 |
| sean-k-mooney | i need libnver 5.1.rc2+ and qemu i would have to check | 16:33 |
| sean-k-mooney | clarkb: ill put it this way the one in fedora 30 is too old | 16:33 |
| sean-k-mooney | well f29 is definetly too old i think f30 was too old when it released but they might have updated | 16:34 |
| clarkb | I've got libvirt 5.6.0 on tumbleweed | 16:34 |
| clarkb | and qemu 4.0 | 16:34 |
| sean-k-mooney | that might actully work but it would proably have the same problem as fedora | 16:35 |
| sean-k-mooney | e.g. its unstable | 16:35 |
| clarkb | and a broken xscreensaver (the joys of rolling releases) | 16:35 |
| clarkb | ya | 16:35 |
| sean-k-mooney | my fedroa job broke because qemu output change and it broke oslo | 16:35 |
| sean-k-mooney | how good is devstack on tumbelweed at this point | 16:36 |
| sean-k-mooney | i ran tubelweed for a bit last year and it was nice but i never trided devstack on it | 16:37 |
| clarkb | I think the biggest issues have been around mirror instability | 16:37 |
| clarkb | they have been reorging and updating their repos recently | 16:37 |
| sean-k-mooney | ah ok. infra mirros it for the gate right | 16:37 |
| sean-k-mooney | but locally it would be more of a proablem i guess | 16:37 |
| *** e0ne has quit IRC | 16:44 | |
| *** spsurya has quit IRC | 16:44 | |
| *** jamesmcarthur has joined #openstack-infra | 16:49 | |
| fungi | well, our mirroring has been problematic for it as well because we don't have an integrity checker for opensuse mirrors so if their official mirrors have broken content we're able to mirror successfully, we'll end up serving that | 16:55 |
| clarkb | ya none of our rsync'd mirrors have integrity checks | 16:57 |
| *** derekh has quit IRC | 17:00 | |
| *** ociuhandu has joined #openstack-infra | 17:04 | |
| *** ociuhandu has quit IRC | 17:09 | |
| *** jamesmcarthur has quit IRC | 17:20 | |
| *** jpena is now known as jpena|off | 17:25 | |
| openstackgerrit | Clark Boylan proposed zuul/zuul-website master: Add Zuul FAQ page https://review.opendev.org/679670 | 17:27 |
| *** jamesmcarthur has joined #openstack-infra | 17:29 | |
| *** tesseract has quit IRC | 17:30 | |
| *** mgoddard has quit IRC | 17:32 | |
| *** mgoddard has joined #openstack-infra | 17:35 | |
| *** ralonsoh has quit IRC | 17:36 | |
| *** kjackal_v2 has quit IRC | 17:40 | |
| *** mgoddard has quit IRC | 17:40 | |
| *** jtomasek has quit IRC | 17:44 | |
| *** zbr|ooo is now known as zbr | 17:53 | |
| *** mgoddard has joined #openstack-infra | 17:53 | |
| *** smarcet has joined #openstack-infra | 17:55 | |
| *** jamesmcarthur has quit IRC | 17:57 | |
| *** jtomasek has joined #openstack-infra | 18:00 | |
| *** gfidente is now known as gfidente|afk | 18:05 | |
| *** smarcet has quit IRC | 18:05 | |
| sean-k-mooney | quick question. do these ssl errors look familar to anyone https://zuul.opendev.org/t/openstack/build/5569d21017f849c59feb2e4a5b736fc3/log/compute/logs/screen-n-cpu.txt.gz?severity=4 | 18:05 |
| sean-k-mooney | i have retriggered the job with tls-proxy: flase to work around it for now | 18:06 |
| sean-k-mooney | is there a special role that needs to be run in multinode jobs to make ssl work? | 18:06 |
| jrosser | When I see things like that it generally means that something that should really connect to an internal (http) andpoint has instead used an external https one | 18:09 |
| clarkb | devstack should use the same root CA for all cert generation on all the nodes (and trust that same CA) | 18:10 |
| clarkb | meaning it should work fine on multinode | 18:10 |
| clarkb | is it possible that the placement service isn't using the devstack CA for its cert? | 18:11 |
| clarkb | other things that may caues that to break are python requests insistence on using its own set of trusted CAs and the job isn't properly telling requests to use the devstack CA too | 18:12 |
| sean-k-mooney | i guess it possible | 18:12 |
| sean-k-mooney | this is the job definition im creating https://review.opendev.org/#/c/679656/4/.zuul.yaml | 18:12 |
| sean-k-mooney | it inherites form tempest-all | 18:12 |
| sean-k-mooney | so im relying on it for 90% of it then tweaking the bits i need to change | 18:13 |
| clarkb | you should not disable dstat or peakmem tracker | 18:13 |
| *** dchen has quit IRC | 18:13 | |
| sean-k-mooney | in generaly yes | 18:13 |
| sean-k-mooney | and ill renable them later | 18:13 |
| sean-k-mooney | but this need to allocate hugepages for the guest l2 guest to use and i was having memory issue previously | 18:14 |
| clarkb | ya but neither of those are a huge memory impact | 18:14 |
| sean-k-mooney | im now usign a bigger node so memory is not an issue | 18:14 |
| clarkb | (as I've noted in my emails to the discuss list there are major memory problems no one seems to care to fix) | 18:14 |
| clarkb | and they aren't the fault of dstat | 18:15 |
| *** e0ne has joined #openstack-infra | 18:15 | |
| sean-k-mooney | ya when this was based on fedoa i gog a failture with dstat which is why i truned it off | 18:15 |
| sean-k-mooney | that was a one off but i didnt want unrelated things failing | 18:16 |
| sean-k-mooney | once its stable ill turn it back on | 18:16 |
| sean-k-mooney | well i can do it in the next version | 18:16 |
| *** kjackal has joined #openstack-infra | 18:17 | |
| sean-k-mooney | clarkb: do the base jobs settup ntpd or chronyd by the way. i have seen time difference cause similar issue before | 18:17 |
| clarkb | sean-k-mooney: comparing what the old devstack-gate multinode setup did to the zuulv3 roles and playbooks in devstack I think the issue is that devstack's zuulv3 jobs don't copy the devstack CA contents to the other hosts and as a result it must generate a second CA? | 18:18 |
| clarkb | sean-k-mooney: yes we configure ntp and force an update in the base jobs | 18:18 |
| sean-k-mooney | clarkb: ok cool. | 18:18 |
| *** pcaruana has quit IRC | 18:19 | |
| sean-k-mooney | regardign the ca i guess i can just add the task as a pre step | 18:19 |
| *** pcaruana has joined #openstack-infra | 18:19 | |
| clarkb | though now that I'm grepping aorund for ntp I see devstack-gate was doing that but don't see that anywhere else | 18:19 |
| clarkb | these two things may have been lost in zuulv3ification? | 18:20 |
| fungi | seems like the missing ca on additional nodes is worth fixing in devstack | 18:20 |
| clarkb | fungi: ya it should be fixed in devstack | 18:20 |
| sean-k-mooney | as i said i have disable the tls-proxy for now to debug. | 18:20 |
| sean-k-mooney | fungi: well in the devstack playbooks/job right | 18:20 |
| clarkb | what should happen is after first stack.sh is run the CA files get copied to the other hosts then when stack.sh runs on them it will use theexisting CA rather tha ncreating a new one | 18:20 |
| sean-k-mooney | rather then devstack code | 18:20 |
| clarkb | sean-k-mooney: in devstack/roles/orchestrate-devstack I think | 18:20 |
| *** smarcet has joined #openstack-infra | 18:21 | |
| sean-k-mooney | clarkb: ya so contoler stacks. then we copy it to all other nodes then stack with the ca | 18:21 |
| sean-k-mooney | i assume we have non legacy multinode tempest jobs? | 18:22 |
| clarkb | ya that needs to happen but I don't see it happening today | 18:22 |
| clarkb | sean-k-mooney: I do not know if anything uses devstack-gate for multinode testing anymore | 18:22 |
| *** smarcet has quit IRC | 18:22 | |
| sean-k-mooney | am i useing devstack gate by the way? | 18:23 |
| clarkb | no you are using the zuulv3 style jobs (you can tell because your devstack config is a dict in the job's vars) | 18:23 |
| sean-k-mooney | well yes im inheriting form https://github.com/openstack/tempest/blob/master/.zuul.yaml#L1-L37 | 18:24 |
| sean-k-mooney | i was wonder ing if that was calling into devstack gate somehow but i did not think it was so good to know i was not wrong | 18:24 |
| clarkb | correct | 18:24 |
| sean-k-mooney | nova still has legacy multinode jobs | 18:24 |
| sean-k-mooney | based on devstack gate | 18:24 |
| sean-k-mooney | specficaly the nova livemigration job | 18:25 |
| sean-k-mooney | part of the reason im writing this is to test numa live migration without extending that legacy job | 18:25 |
| *** rfolco is now known as rfolco|doctor | 18:28 | |
| *** e0ne has quit IRC | 18:46 | |
| AJaeger_ | quick comment before I disappear again: Seeing https://review.opendev.org/679610 for devstack-gate, I wonder whether we should decleare a community goal to switch for U to Zuul v3 jobs and not support U in devstack-gate and other legacy jobs... | 18:56 |
| clarkb | AJaeger_: ++ | 18:57 |
| * AJaeger_ adds to https://etherpad.openstack.org/p/PVG-u-series-goals - but somebody else needs to drive it | 19:03 | |
| tosky | AJaeger_: then we need reviewers for the grande/zuulv3 patches :) | 19:18 |
| *** rkukura has joined #openstack-infra | 19:19 | |
| AJaeger_ | ;) | 19:19 |
| *** pkopec has quit IRC | 19:26 | |
| clarkb | mostof that should be in project repos | 19:32 |
| clarkb | oh grenade? | 19:32 |
| tosky | https://review.opendev.org/#/q/topic:grenade_zuulv3 | 19:37 |
| AJaeger_ | tosky, am I right that https://review.opendev.org/#/c/548936/ is the next change to merge? Is it ready? Then please ping QA folks for reviews... | 19:40 |
| tosky | AJaeger_: gmann is aware and he has that in his TODO list (see the email with his candidacy as PTL) | 19:42 |
| tosky | the only open question is why the test with the heat multinode grenade job fails, but I don't think it's related to the changes] | 19:42 |
| sean-k-mooney | for what its worth assuming i can get time to do it i can try and port the remaining nova legacy jobs | 19:43 |
| sean-k-mooney | we do a few things that are not direclty supported in the run playbook but we could do them in post playbooks | 19:43 |
| sean-k-mooney | greade is proably the bigest legacy job remaining ya | 19:45 |
| *** tobiash has quit IRC | 19:47 | |
| *** sshnaidm|pto has quit IRC | 19:47 | |
| *** tobiash has joined #openstack-infra | 19:48 | |
| *** sshnaidm|pto has joined #openstack-infra | 19:51 | |
| *** logan- has quit IRC | 19:59 | |
| *** gfidente|afk has quit IRC | 20:02 | |
| *** logan- has joined #openstack-infra | 20:02 | |
| *** gfidente has joined #openstack-infra | 20:02 | |
| *** bhavikdbavishi has quit IRC | 20:05 | |
| *** gfidente has quit IRC | 20:14 | |
| *** xenos76 has quit IRC | 20:31 | |
| *** pcaruana has quit IRC | 20:36 | |
| *** smarcet has joined #openstack-infra | 20:44 | |
| *** smarcet has quit IRC | 20:57 | |
| *** smarcet has joined #openstack-infra | 20:59 | |
| *** gfidente has joined #openstack-infra | 21:04 | |
| fungi | #status log logstash-worker16 was rebooted by the provider at 21:11 utc due to a host problem, per provider ticket 190902-ord-0000585 | 21:16 |
| openstackstatus | fungi: finished logging | 21:16 |
| *** gfidente is now known as gfidente|afk | 21:17 | |
| *** jamesmcarthur has joined #openstack-infra | 21:24 | |
| *** tobberydberg has quit IRC | 21:30 | |
| *** tridde has joined #openstack-infra | 21:31 | |
| *** fdegir has quit IRC | 21:31 | |
| *** tobberydberg has joined #openstack-infra | 21:31 | |
| *** trident has quit IRC | 21:32 | |
| *** fdegir has joined #openstack-infra | 21:32 | |
| *** rfolco|doctor is now known as rfolco | 21:35 | |
| *** slaweq has quit IRC | 21:45 | |
| *** hwoarang has quit IRC | 21:53 | |
| *** hwoarang has joined #openstack-infra | 21:53 | |
| *** jamesmcarthur has quit IRC | 22:02 | |
| *** xenos76 has joined #openstack-infra | 22:02 | |
| *** tosky_ has joined #openstack-infra | 22:04 | |
| *** tosky has quit IRC | 22:06 | |
| *** smarcet has left #openstack-infra | 22:06 | |
| *** tosky_ is now known as tosky | 22:09 | |
| *** gfidente|afk has quit IRC | 22:18 | |
| *** jamesmcarthur has joined #openstack-infra | 22:31 | |
| *** smarcet has joined #openstack-infra | 22:40 | |
| *** rcernin has joined #openstack-infra | 22:54 | |
| *** tkajinam has joined #openstack-infra | 22:56 | |
| *** kjackal has quit IRC | 22:57 | |
| *** kjackal has joined #openstack-infra | 22:57 | |
| *** jamesmcarthur has quit IRC | 22:59 | |
| *** jamesmcarthur has joined #openstack-infra | 23:03 | |
| *** srwilkers has quit IRC | 23:04 | |
| *** srwilkers has joined #openstack-infra | 23:04 | |
| *** jamesmcarthur has quit IRC | 23:05 | |
| *** jamesmcarthur has joined #openstack-infra | 23:05 | |
| *** smarcet has quit IRC | 23:09 | |
| *** jamesmcarthur has quit IRC | 23:23 | |
| *** jamesmcarthur has joined #openstack-infra | 23:27 | |
| *** dchen has joined #openstack-infra | 23:30 | |
| *** jamesmcarthur has quit IRC | 23:32 | |
| *** dchen has joined #openstack-infra | 23:32 | |
| *** dchen has quit IRC | 23:32 | |
| *** dchen has joined #openstack-infra | 23:33 | |
| *** owalsh has quit IRC | 23:43 | |
| *** owalsh has joined #openstack-infra | 23:51 | |
| *** jamesmcarthur has joined #openstack-infra | 23:56 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!