| *** rongze has quit IRC | 00:04 | |
| SpamapS | 30878 root 20 0 19500 1884 1144 D 5 0.0 0:08.80 mkfs.ext3 -F -L ephemeral0 /dev/disk/by-path/ip-10.10.1 | 00:09 |
|---|---|---|
| SpamapS | interesting. | 00:09 |
| SpamapS | nova-api is returning 500's for meta-data requests | 00:10 |
| SpamapS | but no idea why | 00:10 |
| SpamapS | 2013-12-04 00:11:36,393.393 993 ERROR nova.api.metadata.handler [req-a4bd8ff8-793e-4f15-93be-1c7572af45e9 None None] Failed to get metadata for ip: 10.10.16.171 | 00:12 |
| SpamapS | 2013-12-04 00:11:36,393.393 993 TRACE nova.api.metadata.handler File "/opt/stack/venvs/nova/local/lib/python2.7/site-packages/nova/network/neutronv2/__init__.py", line 69, in get_client | 00:12 |
| SpamapS | 2013-12-04 00:11:36,393.393 993 TRACE nova.api.metadata.handler raise exceptions.Unauthorized() | 00:12 |
| SpamapS | ok enough debugging for today | 00:12 |
| * SpamapS -> kids | 00:12 | |
| *** boris-42 has quit IRC | 00:14 | |
| lifeless | SpamapS: there was a fix for that | 00:16 |
| lifeless | SpamapS: I thought I pulled it in, may need to refresh nova again. | 00:16 |
| lifeless | SpamapS: which nova-api - the undercloud one? | 00:16 |
| SpamapS | lifeless: right, undercloud nova-api | 00:24 |
| lifeless | so yeah that was fixed | 00:27 |
| lifeless | lets see | 00:27 |
| *** vipul is now known as vipul-away | 00:28 | |
| lifeless | 652620d12f3afe6845e41d9762b52d23f44fd557 | 00:28 |
| lifeless | but its not in /opt/stack/nova now | 00:28 |
| SlickNik | hello good folks. | 00:29 |
| lifeless | SlickNik: allo allo | 00:30 |
| SlickNik | I'm looking at moving the trove dib-image elementes into tripleo-image-elements, and I had two questions. | 00:30 |
| *** cd-undercloud has joined #tripleo | 00:31 | |
| cd-undercloud | ************** overcloud complete status=1 ************ | 00:31 |
| *** cd-undercloud has quit IRC | 00:31 | |
| *** matsuhashi has joined #tripleo | 00:31 | |
| SlickNik | 1. We're still using first-boot.d. I know it's deprecated, and I will fix it over the next few days, but I'm wondering whether it's okay to do the move before fixing it. | 00:31 |
| SlickNik | 2. Is there some way of moving the image-elements over while keeping the git history? I'm afraid if I do a regular gerrit submission, it will all be lost. | 00:32 |
| lifeless | Sure, 1 is fine | 00:32 |
| lifeless | Git history will be lost. | 00:33 |
| SlickNik | Fair enough. 2 is somewhat unfortunate, but understandable. | 00:33 |
| SlickNik | Thanks lifeless! | 00:33 |
| lifeless | SpamapS: updated nova; did an os-collect-config --force --one; restarted tripleo-cd | 00:38 |
| *** vipul-away is now known as vipul | 00:40 | |
| *** lucas-dinner has quit IRC | 00:45 | |
| *** openstack has joined #tripleo | 00:46 | |
| *** openstackgerrit has quit IRC | 00:56 | |
| *** openstackgerrit has joined #tripleo | 00:56 | |
| *** krotscheck has quit IRC | 01:19 | |
| *** julim has quit IRC | 01:22 | |
| *** noslzzp has joined #tripleo | 01:32 | |
| *** kui has joined #tripleo | 01:33 | |
| *** nosnos has joined #tripleo | 01:35 | |
| *** nosnos_ has joined #tripleo | 01:37 | |
| *** nosnos has quit IRC | 01:40 | |
| *** cd-undercloud has joined #tripleo | 01:40 | |
| cd-undercloud | ************** overcloud complete status=1 ************ | 01:40 |
| *** cd-undercloud has quit IRC | 01:40 | |
| *** epim has quit IRC | 01:44 | |
| openstackgerrit | A change was merged to openstack/tripleo-incubator: Add Ironic to setup-undercloud-passwords https://review.openstack.org/59800 | 02:00 |
| dkehn | lifeless: are you going to around for a bit | 02:01 |
| *** rongze has joined #tripleo | 02:01 | |
| *** rongze has joined #tripleo | 02:02 | |
| lifeless | dkehn: ues | 02:13 |
| dkehn | lifeless: I'm in the middle of kids home work, but just wondering if your going to be around later, I'm seeing an interesting situation were the MAC is being chopped 1 octect short in register-nodes | 02:14 |
| lifeless | dkehn: check pending reviews - there is a review there to fix a bug in a wait_for loop. | 02:15 |
| dkehn | yepper that would be it | 02:15 |
| dkehn | yepper hard coded | 02:16 |
| *** rwsu has quit IRC | 02:35 | |
| *** rwsu has joined #tripleo | 02:39 | |
| *** cd-undercloud has joined #tripleo | 02:42 | |
| cd-undercloud | ************** overcloud complete status=1 ************ | 02:42 |
| *** cd-undercloud has quit IRC | 02:42 | |
| SpamapS | ooo | 02:45 |
| SpamapS | the stack actually did complete | 02:46 |
| SpamapS | just took a really long time | 02:46 |
| *** marun has joined #tripleo | 02:46 | |
| SpamapS | | CompletionHandle | 80939 | state changed | CREATE_IN_PROGRESS | 2013-12-04T02:04:04Z | | 02:46 |
| SpamapS | | CompletionCondition | 80962 | state changed | CREATE_COMPLETE | 2013-12-04T02:43:19Z | | 02:46 |
| SpamapS | | notcompute | 80953 | state changed | CREATE_IN_PROGRESS | 2013-12-04T02:04:09Z | | 02:55 |
| SpamapS | | notcompute | 80956 | state changed | CREATE_COMPLETE | 2013-12-04T02:30:44Z | | 02:55 |
| SpamapS | 26 minutes to do two boxes | 02:55 |
| *** ruhe has joined #tripleo | 02:56 | |
| lifeless | ok, so what was the failure? timeout? | 02:57 |
| SpamapS | lifeless: yes | 02:58 |
| *** ruhe has quit IRC | 03:01 | |
| *** vkozhukalov has joined #tripleo | 03:09 | |
| SpamapS | lifeless: it is going now .. might be good to monitor and see how the dd performs.. anything else.. | 03:09 |
| lifeless | instrumentation; we did a design session o nit | 03:10 |
| *** vipul has quit IRC | 03:15 | |
| *** shadower_ has joined #tripleo | 03:16 | |
| *** vipul has joined #tripleo | 03:17 | |
| *** shadower has quit IRC | 03:17 | |
| *** rainya has quit IRC | 03:17 | |
| *** antonym has quit IRC | 03:18 | |
| *** rainya has joined #tripleo | 03:19 | |
| *** phschwartz has quit IRC | 03:19 | |
| *** antonym has joined #tripleo | 03:21 | |
| *** phschwartz has joined #tripleo | 03:22 | |
| *** cd-undercloud has joined #tripleo | 03:46 | |
| cd-undercloud | ************** overcloud complete status=0 ************ | 03:46 |
| *** cd-undercloud has quit IRC | 03:46 | |
| SpamapS | \o/ | 03:52 |
| SpamapS | just under the wire though | 03:53 |
| *** morazi has quit IRC | 03:53 | |
| *** rbrady has quit IRC | 04:04 | |
| *** CaptTofu has quit IRC | 04:15 | |
| *** CaptTofu has joined #tripleo | 04:16 | |
| SpamapS | oohh | 04:24 |
| SpamapS | I didn't realize we had saucy by default now | 04:24 |
| SpamapS | cool! | 04:24 |
| SpamapS | but | 04:24 |
| SpamapS | have to wonder if something regressed performance wise. | 04:24 |
| *** SpamapS changes topic to "Using OpenStack to deploy OpenStack; meetings Tuesday 1900 UTC in #openstack-meeting-alt" | 04:25 | |
| lifeless | SpamapS: well done 86! | 04:30 |
| *** rbrady has joined #tripleo | 04:34 | |
| *** nosnos has joined #tripleo | 04:38 | |
| *** nosnos_ has quit IRC | 04:42 | |
| *** rongze has quit IRC | 04:45 | |
| *** StevenK has joined #tripleo | 04:47 | |
| *** cd-undercloud has joined #tripleo | 04:49 | |
| cd-undercloud | ************** overcloud complete status=1 ************ | 04:49 |
| *** cd-undercloud has quit IRC | 04:49 | |
| SpamapS | nooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooo | 04:49 |
| SpamapS | timed out | 04:50 |
| *** noslzzp has quit IRC | 04:51 | |
| openstackgerrit | Clint "SpamapS" Byrum proposed a change to openstack/tripleo-incubator: Extend devtest_overcloud timeout by 5 minutes https://review.openstack.org/59895 | 04:53 |
| *** rushiagr has joined #tripleo | 04:56 | |
| *** rbrady has quit IRC | 05:15 | |
| *** rongze has joined #tripleo | 05:16 | |
| *** rainya has quit IRC | 05:20 | |
| *** LinuxJedi_ has joined #tripleo | 05:20 | |
| *** michchap_ has joined #tripleo | 05:21 | |
| *** marun has quit IRC | 05:22 | |
| *** jayg|g0n3 has quit IRC | 05:22 | |
| *** stevebaker has quit IRC | 05:22 | |
| *** cwolferh has quit IRC | 05:22 | |
| *** michchap has quit IRC | 05:22 | |
| *** LinuxJedi has quit IRC | 05:22 | |
| *** phschwartz has quit IRC | 05:22 | |
| *** phschwartz has joined #tripleo | 05:22 | |
| *** antonym has quit IRC | 05:22 | |
| *** stevebaker has joined #tripleo | 05:22 | |
| *** marun has joined #tripleo | 05:22 | |
| *** cwolferh has joined #tripleo | 05:22 | |
| *** jayg|g0n3 has joined #tripleo | 05:23 | |
| *** rongze has quit IRC | 05:28 | |
| *** cd-undercloud has joined #tripleo | 05:51 | |
| cd-undercloud | ************** overcloud complete status=1 ************ | 05:51 |
| *** cd-undercloud has quit IRC | 05:51 | |
| lifeless | SpamapS: well, we had our time in the sun | 05:56 |
| SpamapS | lifeless: 20:53 < openstackgerrit> Clint "SpamapS" Byrum proposed a change to openstack/tripleo-incubator: Extend devtest_overcloud timeout by 5 minutes https://review.openstack.org/59895 | 05:56 |
| SpamapS | lifeless: we're _just barely_ missing it | 05:56 |
| lifeless | yah | 05:56 |
| SpamapS | lifeless: It is somewhat confusing why.. that instrumenting work would help.. but for now, I think we are just going to have to live with "it got slower" | 05:57 |
| *** rongze has joined #tripleo | 05:58 | |
| *** rpodolyaka1 has joined #tripleo | 05:59 | |
| rpodolyaka1 | morning tripleo | 06:06 |
| lifeless | rpodolyaka1: o/ | 06:21 |
| *** kui has quit IRC | 06:21 | |
| *** rongze has quit IRC | 06:30 | |
| *** rongze has joined #tripleo | 06:30 | |
| *** nosnos has quit IRC | 06:34 | |
| *** nosnos has joined #tripleo | 06:35 | |
| *** jcooley_ has joined #tripleo | 06:41 | |
| *** akuznetsov has quit IRC | 06:44 | |
| *** jcooley_ has quit IRC | 06:51 | |
| *** boris-42 has joined #tripleo | 06:51 | |
| *** vkozhukalov has quit IRC | 06:51 | |
| *** cd-undercloud has joined #tripleo | 06:52 | |
| cd-undercloud | ************** overcloud complete status=0 ************ | 06:52 |
| *** cd-undercloud has quit IRC | 06:52 | |
| *** marun has quit IRC | 06:57 | |
| *** akuznetsov has joined #tripleo | 06:58 | |
| *** nosnos_ has joined #tripleo | 07:05 | |
| *** nosnos has quit IRC | 07:08 | |
| *** akuznetsov has quit IRC | 07:08 | |
| *** rpodolyaka1 has quit IRC | 07:16 | |
| *** antonym has joined #tripleo | 07:17 | |
| *** rwsu has quit IRC | 07:24 | |
| *** lsmola has joined #tripleo | 07:26 | |
| lsmola | lifeless, hello, are you around? | 07:27 |
| *** jtomasek has joined #tripleo | 07:29 | |
| *** jtomasek has quit IRC | 07:34 | |
| *** pblaho has joined #tripleo | 07:37 | |
| *** jtomasek has joined #tripleo | 07:37 | |
| *** rwsu has joined #tripleo | 07:48 | |
| *** cd-undercloud has joined #tripleo | 07:54 | |
| cd-undercloud | ************** overcloud complete status=1 ************ | 07:54 |
| *** cd-undercloud has quit IRC | 07:54 | |
| *** jprovazn has joined #tripleo | 07:55 | |
| *** bauzas has joined #tripleo | 08:08 | |
| *** marun has joined #tripleo | 08:09 | |
| *** jistr has joined #tripleo | 08:18 | |
| *** nosnos_ has quit IRC | 08:19 | |
| *** nosnos has joined #tripleo | 08:19 | |
| *** GheRivero has quit IRC | 08:22 | |
| *** rdopieralski has joined #tripleo | 08:26 | |
| *** martyntaylor has joined #tripleo | 08:28 | |
| *** vkozhukalov has joined #tripleo | 08:30 | |
| *** matsuhashi has quit IRC | 08:35 | |
| *** matsuhashi has joined #tripleo | 08:36 | |
| *** boris-42 has quit IRC | 08:40 | |
| *** martyntaylor has quit IRC | 08:46 | |
| *** cd-undercloud has joined #tripleo | 08:58 | |
| cd-undercloud | ************** overcloud complete status=1 ************ | 08:58 |
| *** cd-undercloud has quit IRC | 08:58 | |
| *** jcoufal has joined #tripleo | 08:58 | |
| *** martyntaylor has joined #tripleo | 09:00 | |
| Ng | morning | 09:00 |
| *** athomas has joined #tripleo | 09:02 | |
| marios_ | o/ | 09:02 |
| openstackgerrit | A change was merged to openstack/tripleo-incubator: Extend devtest_overcloud timeout by 5 minutes https://review.openstack.org/59895 | 09:04 |
| *** victor_lowther_ has joined #tripleo | 09:06 | |
| *** SlickN1k has joined #tripleo | 09:11 | |
| *** rpodolyaka has quit IRC | 09:12 | |
| *** funzo has quit IRC | 09:12 | |
| *** victor_lowther has quit IRC | 09:12 | |
| *** greghaynes has quit IRC | 09:12 | |
| *** zaro has quit IRC | 09:12 | |
| *** clarkb has quit IRC | 09:12 | |
| *** SlickNik has quit IRC | 09:12 | |
| *** SlickN1k is now known as SlickNik | 09:12 | |
| openstackgerrit | A change was merged to openstack/tripleo-incubator: Test MAC address should be 6 octets. https://review.openstack.org/59812 | 09:12 |
| *** victor_lowther_ is now known as victor_lowther | 09:12 | |
| *** akuznetsov has joined #tripleo | 09:14 | |
| *** derekh has joined #tripleo | 09:15 | |
| *** zaro has joined #tripleo | 09:18 | |
| *** greghaynes has joined #tripleo | 09:18 | |
| *** clarkb has joined #tripleo | 09:19 | |
| *** rpodolyaka has joined #tripleo | 09:20 | |
| *** max_lobur_afk is now known as max_lobur | 09:24 | |
| *** lucasagomes has joined #tripleo | 09:31 | |
| *** markmc has joined #tripleo | 09:42 | |
| *** jtomasek has quit IRC | 09:43 | |
| *** jtomasek has joined #tripleo | 09:43 | |
| *** boris-42 has joined #tripleo | 09:46 | |
| *** akrivoka has joined #tripleo | 09:47 | |
| openstackgerrit | Derek Higgins proposed a change to openstack-infra/tripleo-ci: Add element to install a testenv worker https://review.openstack.org/58305 | 09:48 |
| openstackgerrit | Derek Higgins proposed a change to openstack-infra/tripleo-ci: Add script to install a testenv client https://review.openstack.org/58306 | 09:48 |
| *** cd-undercloud has joined #tripleo | 09:59 | |
| cd-undercloud | ************** overcloud complete status=0 ************ | 09:59 |
| *** cd-undercloud has quit IRC | 09:59 | |
| Ng | ooh, status=0 | 10:02 |
| rdopieralski | \o/ | 10:02 |
| *** nosnos_ has joined #tripleo | 10:03 | |
| openstackgerrit | Jun Jie Nan proposed a change to openstack/diskimage-builder: Fix no busybox symlinks issue on rhel https://review.openstack.org/59940 | 10:06 |
| *** nosnos has quit IRC | 10:06 | |
| *** matsuhashi has quit IRC | 10:22 | |
| *** athomas has quit IRC | 10:37 | |
| *** funzo has joined #tripleo | 10:43 | |
| *** athomas has joined #tripleo | 10:44 | |
| *** matsuhashi has joined #tripleo | 10:45 | |
| *** jistr has quit IRC | 10:46 | |
| *** rongze has quit IRC | 10:53 | |
| *** cd-undercloud has joined #tripleo | 11:02 | |
| cd-undercloud | ************** overcloud complete status=0 ************ | 11:02 |
| *** cd-undercloud has quit IRC | 11:02 | |
| *** jistr has joined #tripleo | 11:09 | |
| *** rongze has joined #tripleo | 11:11 | |
| *** jergerber has quit IRC | 11:43 | |
| *** rongze has quit IRC | 11:45 | |
| *** rongze has joined #tripleo | 11:51 | |
| *** max_lobur is now known as max_lobur_afk | 11:57 | |
| *** cd-undercloud has joined #tripleo | 12:05 | |
| cd-undercloud | ************** overcloud complete status=0 ************ | 12:05 |
| *** cd-undercloud has quit IRC | 12:05 | |
| *** CaptTofu has quit IRC | 12:30 | |
| *** CaptTofu has joined #tripleo | 12:31 | |
| *** jprovazn has quit IRC | 12:37 | |
| *** rbrady has joined #tripleo | 12:39 | |
| *** shadower_ is now known as shadower | 12:43 | |
| *** flwang has joined #tripleo | 12:43 | |
| jog0 | lifeless: turns out the gate is hurting again :/ | 12:48 |
| jog0 | 23% failure | 12:48 |
| flwang | morning guys | 12:49 |
| flwang | may I ask a stupid question about TripleO? | 12:49 |
| *** max_lobur_afk is now known as max_lobur | 12:52 | |
| *** jcoufal has quit IRC | 12:52 | |
| *** akuznetsov has quit IRC | 12:53 | |
| *** akuznetsov has joined #tripleo | 12:56 | |
| *** jcoufal has joined #tripleo | 12:57 | |
| *** morazi has joined #tripleo | 12:57 | |
| *** jprovazn has joined #tripleo | 12:58 | |
| *** lucasagomes is now known as lucas-hungry | 12:59 | |
| rpodolyaka | flwang: sure you may :) | 13:02 |
| *** matsuhashi has quit IRC | 13:03 | |
| *** dprince has joined #tripleo | 13:03 | |
| *** panda_ has joined #tripleo | 13:03 | |
| *** panda_ is now known as panda | 13:04 | |
| *** cd-undercloud has joined #tripleo | 13:06 | |
| cd-undercloud | ************** overcloud complete status=42 ************ | 13:06 |
| *** cd-undercloud has quit IRC | 13:06 | |
| *** nosnos_ has quit IRC | 13:07 | |
| *** nosnos has joined #tripleo | 13:08 | |
| *** akuznetsov has quit IRC | 13:13 | |
| *** jdob has joined #tripleo | 13:14 | |
| *** CaptTofu has quit IRC | 13:15 | |
| *** CaptTofu has joined #tripleo | 13:16 | |
| *** akuznetsov has joined #tripleo | 13:16 | |
| *** rushiagr has quit IRC | 13:31 | |
| *** matsuhashi has joined #tripleo | 13:32 | |
| flwang | is it possible to leverage TripleO to do some auto scaling? as we said on the wiki https://wiki.openstack.org/wiki/TripleO the cloud can be 'scaling down to as few as 2 machines'. thanks | 13:38 |
| *** noslzzp has joined #tripleo | 13:38 | |
| *** noslzzp has quit IRC | 13:39 | |
| *** noslzzp has joined #tripleo | 13:39 | |
| *** nosnos has quit IRC | 13:39 | |
| *** CaptTofu has quit IRC | 13:39 | |
| jdob | is there a way to see open code reviews for all of the tripleo-related projects, or do I need to do queries on tripleo-incubator, tripleo-image-elements, etc. individually? | 14:02 |
| markmc | jdob, see https://wiki.openstack.org/wiki/TripleO#Review_team | 14:03 |
| *** lucas-hungry is now known as lucasagomes | 14:03 | |
| jdob | i'm not familiar enough with the search syntax on the review site to know if it supports any so-- cool, will check there, thanks markmc | 14:03 |
| markmc | np :) | 14:03 |
| jdob | ah perfect, that's exactly what I was trying to do; I was missing the () around my attempt at an OR query | 14:03 |
| *** julim has joined #tripleo | 14:05 | |
| rpodolyaka | jog0: hey, this might be interesting for you https://bugs.launchpad.net/openstack-ci/+bug/1216851/comments/6 | 14:06 |
| uvirtbot | Launchpad bug 1216851 in nova "nova unit tests occasionally fail migration tests for mysql and postgres" [Undecided,Confirmed] | 14:06 |
| jog0 | rpodolyaka: awesome sauce -- if we can't bump that number up for specific tests lets bump it up accross the board | 14:07 |
| jog0 | rpodolyaka: can you look at a working py27 test and see how long migrations usually take. are we alyways close to the timelimit or do we have some freak outliers | 14:08 |
| rdopieralski | > | 14:08 |
| rdopieralski | > Thanks, | 14:08 |
| rdopieralski | > | 14:08 |
| rdopieralski | > Sescia | 14:08 |
| rdopieralski | sorry | 14:08 |
| rdopieralski | misclick | 14:08 |
| *** cd-undercloud has joined #tripleo | 14:09 | |
| cd-undercloud | ************** overcloud complete status=0 ************ | 14:09 |
| *** cd-undercloud has quit IRC | 14:09 | |
| rpodolyaka | jog0: running test_walk_versions() on my machine, will tell you the results soon | 14:12 |
| *** jayg|g0n3 is now known as jayg | 14:13 | |
| rpodolyaka | flwang: sorry, missed you message. I would say, it will be possible to autoscale both overcloud and undercloud, but not right now | 14:14 |
| *** rongze has quit IRC | 14:14 | |
| *** morazi has quit IRC | 14:15 | |
| jog0 | cool, you can also look at the logs from a working gate job | 14:16 |
| *** rongze has joined #tripleo | 14:18 | |
| *** morazi has joined #tripleo | 14:26 | |
| rpodolyaka | jog0: so these are results for my machine http://paste.openstack.org/show/54424/ | 14:30 |
| rpodolyaka | jog0: checking gate logs | 14:31 |
| rpodolyaka | jog0: interesting, I took 3 random successful runs of gate jobs in nova and results here vary greatly - e.g. test_mysql_opportunistically from 41s to 67s, test_mysql_opportunistically - 93s to 106s | 14:38 |
| rpodolyaka | jog0: the latter was supposed to be test_postgresql_opportunistically | 14:38 |
| rpodolyaka | jog0: oh, so for test_postgresql_opportunistically the best result I found was 38s. So the results seem to depend heavily on the load of hosts running CI nodes (and postresql are often fairly close to 160s timeout we have now) | 14:42 |
| *** matty_dubs|gone is now known as matty_dubs | 14:44 | |
| *** bauzas has quit IRC | 14:56 | |
| jog0 | rpodolyaka: cool it sounds like we should bump that timeout up then | 14:57 |
| rpodolyaka | jog0: yeah, I wonder how much we should bump it though | 14:57 |
| *** jtomasek has quit IRC | 14:57 | |
| *** fungi has joined #tripleo | 14:58 | |
| *** ChanServ sets mode: +v fungi | 14:58 | |
| fungi | jog0: sez someone might know why nova db migrations suddenly got slooooow | 14:58 |
| jog0 | rpodolyaka: ^ fungi | 14:58 |
| fungi | jog0: you mentioned a 160-second timeout. is that per migration step, or for the list of migrations in aggregate? | 14:59 |
| rpodolyaka | fungi: all migrations | 14:59 |
| fungi | oh, the 160sec timeout is a testr unit timeout, and the migrations are performed together as one test | 15:00 |
| rpodolyaka | fungi: right | 15:00 |
| fungi | so devs are constantly adding new migration steps, and the timeout is now coming into sight on some runs as a result | 15:00 |
| fungi | i assume the recommended solution for that is to perform each migration step as a separate test, with a known start and end state? | 15:01 |
| *** jtomasek has joined #tripleo | 15:02 | |
| fungi | that will presumably improve parallelization/throughput for the tests anyway | 15:02 |
| *** jistr has quit IRC | 15:02 | |
| fungi | assuming the migrations are able to be properly partitioned so that they use dedicated per-migration databases and don't step on one anothers' toes | 15:03 |
| jog0 | fungi: good idea, rpodolyaka is that possible? | 15:03 |
| *** jistr has joined #tripleo | 15:04 | |
| *** jistr is now known as jistr|mtg | 15:04 | |
| * rpodolyaka thinking | 15:04 | |
| mordred | you'd have to get the start state for the migration test | 15:04 |
| mordred | the only currently known way to get a start state for a given migration is to run the migrations that come befor eit | 15:05 |
| rpodolyaka | yeah | 15:05 |
| mordred | usually at the beginning of the cycle, someone replaces the migrations with a single rollup migration | 15:05 |
| rpodolyaka | so this seems to be a bad idea | 15:05 |
| *** cd-undercloud has joined #tripleo | 15:05 | |
| cd-undercloud | ************** overcloud complete status=0 ************ | 15:05 |
| *** cd-undercloud has quit IRC | 15:05 | |
| rpodolyaka | and we already have problems with tests accessing the same db which are run in parallel | 15:05 |
| mordred | we shoudl really have schema-per-test | 15:06 |
| rpodolyaka | +1 | 15:06 |
| * mordred needs to write the nova patch to use drizzle for unitttests instead of mysql | 15:06 | |
| *** jtomasek has quit IRC | 15:07 | |
| rpodolyaka | mordred: fungi: jog0: https://etherpad.openstack.org/p/db-schemas-provisioning-on-ci | 15:07 |
| rpodolyaka | mordred: fungi: jog0: so we have a POC implemented in Nova/oslo-incubator, but clarkb said mysql openstack_citest user didn't have enough rights to create/drop schemas | 15:09 |
| *** john-n-seattle1 has quit IRC | 15:09 | |
| mordred | that's right. | 15:11 |
| fungi | rpodolyaka: i believe that is true for mysql but not for postgres | 15:11 |
| mordred | you can't give create/drop schema in mysql without also giving admin rights, iirc | 15:12 |
| mordred | which is why I'm mainly suggesting replacing mysql with drizzle in our testing, since you can just stsart drizzle with no auth plugins and there will be no auth | 15:12 |
| *** jtomasek has joined #tripleo | 15:12 | |
| * rpodolyaka googling | 15:12 | |
| mordred | and that way we could stop having a pre-existing db, and instead start the db as part of a test fixture | 15:12 |
| fungi | and postgres has fine-grained permissions in that area already (we may even already set them?) which i believe allow us to grant the control we need safely. at least that's what i recall from prior discussions | 15:13 |
| * mordred used to be core on drizzle | 15:13 | |
| mordred | yup | 15:13 |
| *** bauzas has joined #tripleo | 15:14 | |
| jog0 | so then short term lets bump up the timeout past 160? | 15:20 |
| jog0 | mikal: ^ where is your make dbs faster 'turbo hipster' tool when we need it | 15:21 |
| jog0 | err make sure we have no slow migrations | 15:21 |
| *** lucasagomes_ has joined #tripleo | 15:22 | |
| *** matsuhashi has quit IRC | 15:23 | |
| jog0 | rpodolyaka: sounds like we have along term solution, just need a short term one too | 15:23 |
| *** lucasagomes has quit IRC | 15:24 | |
| rpodolyaka | jog0: yeah, and OS_TEST_TIMEOUT seems to be the only one. I wish there was a way to increase it for specific tests | 15:24 |
| jog0 | lifeless: ^ | 15:24 |
| jog0 | rpodolyaka: push a patch up to change, and say that you ideally want a per test change and hope someone knows | 15:25 |
| jog0 | such as lifeless | 15:25 |
| rpodolyaka | jog0: ack | 15:25 |
| jog0 | self.useFixture(fixtures.Timeout(test_timeout, gentle=True)) | 15:25 |
| jog0 | I wonder if setting a different value will override that | 15:26 |
| *** lucasagomes_ is now known as lucasagomes | 15:26 | |
| *** jcooley_ has joined #tripleo | 15:30 | |
| openstackgerrit | James Slagle proposed a change to openstack/tripleo-image-elements: Add diskimage-builder element. https://review.openstack.org/59361 | 15:32 |
| SpamapS | flwang: auto, not just yet, but scaling, yes. | 15:39 |
| flwang | SpamapS: got, so do you think it's a good food to try? | 15:39 |
| *** john-n-seattle1 has joined #tripleo | 15:42 | |
| SpamapS | flwang: the problem with auto scaling is that you cannot decide which machines to remove. | 15:43 |
| flwang | SpamapS: you mean maybe there are something are running on it? | 15:44 |
| *** john-n-seattle1 has quit IRC | 15:46 | |
| *** rongze has quit IRC | 15:46 | |
| SpamapS | flwang: right. It is great for things like API nodes, but not so much for compute nodes. :) | 15:50 |
| flwang | SpamapS: yep, but actually, what I want to do is autoscaling the OpenStack cloud controller, such as the api services, mq, db, etc | 15:51 |
| *** max_lobur is now known as max_lobur_afk | 15:51 | |
| SpamapS | flwang: mq and db are also problematic | 15:52 |
| SpamapS | flwang: if you are in a degraded state already in your HA db/mq setup, you must not scale down until you have recovered. | 15:52 |
| *** jistr|mtg has quit IRC | 15:53 | |
| *** vkozhukalov has quit IRC | 15:53 | |
| flwang | SpamapS: good point, so how about the api services? | 15:53 |
| SpamapS | flwang: I think Heat autoscaling should be able to automate these cases, but it definitely does not today. | 15:53 |
| SpamapS | flwang: anything stateless is crazy easy. :) | 15:54 |
| flwang | SpamapS: yep, I will leverage Heat as well | 15:54 |
| flwang | SpamapS: yep, state is the key point | 15:54 |
| flwang | SpamapS: so overall, do you think it's worthy to try? | 15:55 |
| SpamapS | flwang: if you have need to scale out the API services, I'd happily accept a set of tripleo-heat-templates which make that possible. | 15:56 |
| SpamapS | flwang: for the other pieces, Heat is undergoing a lot of autoscaling changes that will perhaps make this possible later on. | 15:57 |
| flwang | SpamapS: cool, thanks for all your valuable suggestion | 15:57 |
| flwang | SpamapS: really helpful | 15:57 |
| *** cd-undercloud has joined #tripleo | 15:58 | |
| cd-undercloud | ************** overcloud complete status=0 ************ | 15:58 |
| *** cd-undercloud has quit IRC | 15:58 | |
| openstackgerrit | Yuriy Zveryanskyy proposed a change to openstack/diskimage-builder: Add deploy ramdisk element for Ironic https://review.openstack.org/59770 | 15:59 |
| *** jistr has joined #tripleo | 15:59 | |
| *** jistr is now known as jistr|mtg | 16:00 | |
| *** rongze has joined #tripleo | 16:02 | |
| *** john-n-seattle1 has joined #tripleo | 16:03 | |
| *** funzo_ has joined #tripleo | 16:08 | |
| *** funzo has quit IRC | 16:08 | |
| *** rdopieralski has quit IRC | 16:16 | |
| *** jcoufal has quit IRC | 16:16 | |
| *** rbrady has quit IRC | 16:16 | |
| *** funzo_ has quit IRC | 16:17 | |
| mikal | jog0: so, I am hoping to turn turbo hipster on today | 16:17 |
| mikal | jog0: just fighting final config stuff | 16:17 |
| mikal | jog0: but I have been abandoned to do it all myself, and I'm not very smart | 16:18 |
| *** funzo has joined #tripleo | 16:20 | |
| *** rbrady has joined #tripleo | 16:26 | |
| *** rushiagr has joined #tripleo | 16:32 | |
| *** UtahDave has joined #tripleo | 16:35 | |
| jog0 | mikal: cool, I ask because we are seeing the py27 nova tests failing due to slow migration tests | 16:37 |
| *** CaptTofu has joined #tripleo | 16:39 | |
| mikal | jog0: ? | 16:40 |
| mikal | jog0: as in the unit test times out during a snake walk? | 16:40 |
| SpamapS | you know | 16:41 |
| SpamapS | TripleO deployment has gotten slower of late | 16:41 |
| jog0 | https://bugs.launchpad.net/openstack-ci/+bug/1216851 + scroll back | 16:41 |
| uvirtbot | Launchpad bug 1216851 in nova "nova unit tests occasionally fail migration tests for mysql and postgres" [Undecided,Confirmed] | 16:41 |
| SpamapS | perhaps because of slow migrations (with empty databases no less!) | 16:41 |
| jog0 | mikal: so yes snake walk | 16:41 |
| SpamapS | I wonder if we could make a job which automates the roll up. | 16:42 |
| jog0 | SpamapS: maybe we can blame the new version of sqlalchemy-migrate that we released | 16:42 |
| jog0 | maybe its now just slower for some reason | 16:42 |
| mikal | jog0: so that's going to be systemic. Its not a simple fix. | 16:42 |
| jog0 | SpamapS: shouldn't be too hard to confirm | 16:42 |
| SpamapS | jog0: the migrations only got a little slower.. I don't have instrumentation to confirm when they got longer or how much longer they got. | 16:42 |
| SpamapS | jog0: though they've always been slower than they need to be. | 16:43 |
| jog0 | mikal: yeah I know :( unless there is an easy explanation like new sqlalchemy-migrate (FWIW I am just hoping, I don't actually thing that is the problem) | 16:43 |
| mikal | jog0: not that I am aware of, but I am swamped because I'm travelleing and therefore in meetings all the time | 16:43 |
| *** jcooley_ has quit IRC | 16:43 | |
| * jog0 wanders off back to the Hanukkah party | 16:44 | |
| jog0 | anyway we have a short term workaround . bump up the timeout then we can dig into this without it being a gate issue | 16:44 |
| mikal | Ok, cool | 16:45 |
| mikal | I do have a list of slow migrations I cn share | 16:45 |
| mikal | So that might be a good starting point | 16:45 |
| *** UtahDave has quit IRC | 16:47 | |
| *** shakayumi has joined #tripleo | 16:48 | |
| *** michchap_ has quit IRC | 16:48 | |
| *** cd-undercloud has joined #tripleo | 16:49 | |
| cd-undercloud | ************** overcloud complete status=0 ************ | 16:49 |
| *** cd-undercloud has quit IRC | 16:49 | |
| *** marun has quit IRC | 16:50 | |
| *** cwolferh_ has joined #tripleo | 16:52 | |
| *** jistr|mtg is now known as jistr | 16:54 | |
| *** cwolferh has quit IRC | 16:55 | |
| *** LinuxJedi_ is now known as LinuxJedi | 16:59 | |
| *** akuznetsov has quit IRC | 17:05 | |
| *** rongze has quit IRC | 17:05 | |
| *** rongze has joined #tripleo | 17:07 | |
| *** jcooley_ has joined #tripleo | 17:08 | |
| *** jcooley_ has quit IRC | 17:15 | |
| openstackgerrit | Mark McLoughlin proposed a change to openstack/diskimage-builder: source-repositories: log the repo we're cloning from https://review.openstack.org/60035 | 17:19 |
| *** jistr has quit IRC | 17:19 | |
| *** rushiagr has quit IRC | 17:22 | |
| *** rushiagr has joined #tripleo | 17:22 | |
| *** jdob_ has joined #tripleo | 17:26 | |
| *** pblaho has quit IRC | 17:27 | |
| *** jdob has quit IRC | 17:27 | |
| *** martyntaylor has quit IRC | 17:36 | |
| *** jdob_ has quit IRC | 17:38 | |
| *** jdob has joined #tripleo | 17:38 | |
| *** vkozhukalov has joined #tripleo | 17:39 | |
| *** matty_dubs is now known as matty_dubs|lunch | 17:42 | |
| *** cd-undercloud has joined #tripleo | 17:47 | |
| cd-undercloud | ************** overcloud complete status=42 ************ | 17:47 |
| *** cd-undercloud has quit IRC | 17:47 | |
| openstackgerrit | A change was merged to openstack/tripleo-image-elements: Add diskimage-builder element. https://review.openstack.org/59361 | 17:48 |
| *** akuznetsov has joined #tripleo | 17:50 | |
| *** derekh has quit IRC | 17:50 | |
| *** michchap has joined #tripleo | 17:56 | |
| *** martyntaylor has joined #tripleo | 18:03 | |
| *** michchap has quit IRC | 18:03 | |
| *** akuznetsov has quit IRC | 18:03 | |
| * Ng dinners | 18:06 | |
| *** 16WABNAJZ has joined #tripleo | 18:11 | |
| *** 16WABNAJZ has quit IRC | 18:17 | |
| *** akrivoka has quit IRC | 18:19 | |
| *** lucasagomes has quit IRC | 18:25 | |
| * SpamapS polishes off a little utility to test the metadata + moustache templates embedded in elements | 18:31 | |
| *** matty_dubs|lunch is now known as matty_dubs | 18:31 | |
| *** athomas has quit IRC | 18:31 | |
| *** rbrady has quit IRC | 18:33 | |
| *** markmc has quit IRC | 18:34 | |
| lifeless | morning | 18:38 |
| lifeless | Ng: hey so | 18:38 |
| lifeless | Ng: what do you think of us capturing the build environment to a separate .build file? | 18:39 |
| *** akuznetsov has joined #tripleo | 18:39 | |
| *** cd-undercloud has joined #tripleo | 18:40 | |
| cd-undercloud | ************** overcloud complete status=0 ************ | 18:40 |
| *** cd-undercloud has quit IRC | 18:40 | |
| openstackgerrit | A change was merged to openstack/diskimage-builder: Add deploy ramdisk element for Ironic https://review.openstack.org/59770 | 18:40 |
| *** jprovazn has quit IRC | 18:42 | |
| *** jprovazn has joined #tripleo | 18:44 | |
| *** rushiagr has quit IRC | 18:45 | |
| openstackgerrit | James Slagle proposed a change to openstack/diskimage-builder: Default name for ramdisks to deploy. https://review.openstack.org/60046 | 18:46 |
| openstackgerrit | Clint "SpamapS" Byrum proposed a change to openstack/os-apply-config: Log parsing errors from pystache as errors https://review.openstack.org/60047 | 18:47 |
| dprince | slagle: you beat me to approving that ironic commit which has an issue for RHEL/Fedora | 18:49 |
| * dprince volunteers slagle to fix it now :) | 18:50 | |
| dprince | slagle: see here https://review.openstack.org/#/c/59770/ | 18:51 |
| *** krotscheck has joined #tripleo | 18:55 | |
| slagle | dprince: see my reply :) | 18:55 |
| *** michchap has joined #tripleo | 18:56 | |
| dprince | slagle: ah. We're good then. | 18:59 |
| dprince | slagle: That busybox context always gets me (and has many times) | 18:59 |
| dprince | slagle: I'm always like... this runs fine for me. Why won't busybox run it! | 18:59 |
| slagle | yea :) | 19:00 |
| SpamapS | wow, go us.. we have not created any invalid yaml or json in README.md in tripleo-image-elements | 19:00 |
| *** michchap has quit IRC | 19:01 | |
| dprince | SpamapS: hi, question for you on https://review.openstack.org/#/c/59297/ | 19:03 |
| dprince | SpamapS: do you want me to go on and implement your upstart suggestions? Or would you prefer to do them? | 19:04 |
| SpamapS | dprince: if you are comfortable implementing it, I think I spelled it out well enough.. | 19:04 |
| SpamapS | dprince: totally happy for somebody else to do it :) | 19:04 |
| dprince | SpamapS: and we believe this will make the lifeless -2 go away | 19:04 |
| dprince | SpamapS: well, not on this patch... but later in the series he -2'd me. His comments are on the etherpad here https://etherpad.openstack.org/p/tripleo-late-start-services | 19:05 |
| * dprince just wants everyone to weigh in before he does the work | 19:07 | |
| SpamapS | dprince: right we discussed a bit yesterday and then I thought about it. | 19:08 |
| dprince | SpamapS: okay. The approaches seemed similar. I wasn't sure you both had talked... | 19:08 |
| SpamapS | dprince: upstart was kind of my thing at Canonical, so I am pretty confident this will work. I've wanted this built in to upstart since forever actually. :) | 19:08 |
| dprince | SpamapS: On the systemd side I actually think my existing approach is sufficient. | 19:09 |
| SpamapS | dprince: agreed | 19:09 |
| SpamapS | because systemd has a sane enable/disable mechanism IIRC | 19:09 |
| dprince | SpamapS: cool. Let me see if I can whip this up then. | 19:10 |
| openstackgerrit | A change was merged to openstack-infra/tripleo-ci: Add element to install a testenv worker https://review.openstack.org/58305 | 19:24 |
| *** rpodolyaka1 has joined #tripleo | 19:24 | |
| openstackgerrit | Dan Prince proposed a change to openstack-infra/tripleo-ci: Drop notcompute element. https://review.openstack.org/59112 | 19:25 |
| openstackgerrit | A change was merged to openstack-infra/tripleo-ci: Add script to install a testenv client https://review.openstack.org/58306 | 19:25 |
| *** vkozhukalov has quit IRC | 19:26 | |
| openstackgerrit | Clint "SpamapS" Byrum proposed a change to openstack/tripleo-image-elements: Create tests for example metadata and templates https://review.openstack.org/60054 | 19:27 |
| openstackgerrit | Clint "SpamapS" Byrum proposed a change to openstack/tripleo-image-elements: Add script to test all example metadata https://review.openstack.org/60055 | 19:27 |
| openstackgerrit | Clint "SpamapS" Byrum proposed a change to openstack/tripleo-image-elements: Run tests on elements in tox.ini https://review.openstack.org/60056 | 19:27 |
| openstackgerrit | Clint "SpamapS" Byrum proposed a change to openstack/tripleo-image-elements: Create tests for example metadata and templates https://review.openstack.org/60054 | 19:33 |
| openstackgerrit | Clint "SpamapS" Byrum proposed a change to openstack/tripleo-image-elements: Add script to test all example metadata https://review.openstack.org/60055 | 19:33 |
| openstackgerrit | Clint "SpamapS" Byrum proposed a change to openstack/tripleo-image-elements: Run tests on elements in tox.ini https://review.openstack.org/60056 | 19:33 |
| *** rbrady has joined #tripleo | 19:39 | |
| openstackgerrit | James Slagle proposed a change to openstack/diskimage-builder: Default name for ramdisks to image. https://review.openstack.org/60046 | 19:42 |
| *** bauzas has quit IRC | 19:48 | |
| *** spzala has joined #tripleo | 19:49 | |
| openstackgerrit | Dan Prince proposed a change to openstack/tripleo-incubator: Use set-os-type to set default NODE_DIST https://review.openstack.org/58426 | 19:56 |
| *** michchap has joined #tripleo | 19:57 | |
| *** jdob has quit IRC | 20:00 | |
| *** michchap has quit IRC | 20:02 | |
| *** jdob has joined #tripleo | 20:03 | |
| *** boris-42 has quit IRC | 20:04 | |
| *** bauzas has joined #tripleo | 20:06 | |
| *** derekh has joined #tripleo | 20:10 | |
| lifeless | ok-> town for this thing, bbia few hours | 20:11 |
| dprince | pleia2/derekh: you guys ready to chat then? | 20:16 |
| derekh | dprince: ready | 20:16 |
| pleia2 | dprince: yep | 20:16 |
| *** boris-42 has joined #tripleo | 20:22 | |
| *** jtomasek has quit IRC | 20:28 | |
| openstackgerrit | James Slagle proposed a change to openstack/diskimage-builder: Default name for ramdisks to image. https://review.openstack.org/60046 | 20:31 |
| *** derekh has quit IRC | 20:46 | |
| dprince | ssh: connect to host cd-undercloud.tripleo.org port 22: Connection timed out | 20:47 |
| dprince | Can anyone else connect to the CD environment? | 20:47 |
| *** dprince has quit IRC | 20:50 | |
| *** vipul is now known as vipul-away | 20:51 | |
| Ng | d0ugal: hrm, no | 20:52 |
| SpamapS | it's down for me too | 20:52 |
| Ng | on it via serial, checking for anything obviously full of fail | 20:53 |
| Ng | it doesn't immediately be able to ping anything on the public or private IP ranges | 20:53 |
| SpamapS | it may be mellanox fail again | 20:53 |
| Ng | dmesg is full of disk errors :/ | 20:53 |
| SpamapS | Ng: ignore loop disk errors | 20:54 |
| Ng | lots of sdb | 20:54 |
| SpamapS | lovely | 20:54 |
| Ng | aaaaaand mellanox cmd_pending failures | 20:54 |
| Ng | SpamapS: can we just rmmod/modprobe the mlx4 KOs? or is this a reboot? | 20:55 |
| Ng | meh, rmmod doesn't actually work | 21:01 |
| *** Ng changes topic to "CRITICAL: undercloud mlx4 driver failed again. Using OpenStack to deploy OpenStack; meetings Tuesday 1900 UTC in #openstack-meeting-alt" | 21:01 | |
| openstackgerrit | A change was merged to openstack/tripleo-incubator: Use set-os-type to set default NODE_DIST https://review.openstack.org/58426 | 21:01 |
| Ng | rather than just reboot, I'll hold things as they are, for someone to be around, who knows how to restart this without rebooting | 21:02 |
| Ng | I don't want to put us back to square one on the rebooting thing, having only just dug ourselves out of that ;) | 21:02 |
| *** morazi has quit IRC | 21:06 | |
| *** sballe has joined #tripleo | 21:15 | |
| *** noslzzp has quit IRC | 21:15 | |
| openstackgerrit | Chris Jones proposed a change to openstack/tripleo-incubator: Assert undercloud auth before building overcloud. https://review.openstack.org/59237 | 21:16 |
| *** jprovazn has quit IRC | 21:17 | |
| SpamapS | Ng: the rmmod often locks up for a while | 21:18 |
| SpamapS | Ng: note that I found a newer version of the mlx driver .. haven't tried it yet though. | 21:19 |
| Ng | k, I'll leave it running for a bit, but I'm not going to be around for very much longer | 21:19 |
| *** epim has joined #tripleo | 21:20 | |
| *** morazi has joined #tripleo | 21:24 | |
| *** julim has quit IRC | 21:25 | |
| *** vipul-away is now known as vipul | 21:25 | |
| SpamapS | Ng: you on via conman? | 21:25 |
| SpamapS | Ng: I'll take over. | 21:26 |
| openstackgerrit | Clint "SpamapS" Byrum proposed a change to openstack/tripleo-image-elements: Add an ntp element https://review.openstack.org/60079 | 21:26 |
| Ng | SpamapS: yeah I just logged in via conman (0017) and it refused to rmmod at all | 21:26 |
| Ng | oh, I wonder if my previous process is still running from before I logged out | 21:26 |
| openstackgerrit | Clint "SpamapS" Byrum proposed a change to openstack/tripleo-image-elements: Add script to test all example metadata https://review.openstack.org/60055 | 21:30 |
| openstackgerrit | Clint "SpamapS" Byrum proposed a change to openstack/tripleo-image-elements: Create tests for example metadata and templates https://review.openstack.org/60054 | 21:33 |
| openstackgerrit | Clint "SpamapS" Byrum proposed a change to openstack/tripleo-image-elements: Add script to test all example metadata https://review.openstack.org/60055 | 21:33 |
| openstackgerrit | Clint "SpamapS" Byrum proposed a change to openstack/tripleo-image-elements: Run tests on elements in tox.ini https://review.openstack.org/60056 | 21:34 |
| jdob | when does the bot post about a change, each patch set on a review? | 21:35 |
| rpodolyaka1 | yep | 21:37 |
| rpodolyaka1 | and when a change is merged | 21:37 |
| jdob | gotcha. threw me off when I kept seeing the same review mentioned multiple times | 21:38 |
| *** spzala has quit IRC | 21:43 | |
| *** rbrady has quit IRC | 21:46 | |
| *** lsmola has quit IRC | 21:47 | |
| *** rongze has quit IRC | 21:49 | |
| *** CaptTofu has quit IRC | 21:53 | |
| *** rbrady has joined #tripleo | 21:55 | |
| *** rbrady has left #tripleo | 21:55 | |
| *** noslzzp has joined #tripleo | 22:10 | |
| *** shakayumi has quit IRC | 22:12 | |
| *** jdob has quit IRC | 22:16 | |
| *** rongze has joined #tripleo | 22:19 | |
| SpamapS | rmmod mlx4_en finally finished, now waiting for mlx4_core | 22:21 |
| SpamapS | back up | 22:23 |
| *** panda has quit IRC | 22:24 | |
| *** rongze has quit IRC | 22:28 | |
| openstackgerrit | Elizabeth Krumbach Joseph proposed a change to openstack/tripleo-incubator: Add Elizabeth Krumbach Joseph to tripleo-cd-admins https://review.openstack.org/60097 | 22:29 |
| lifeless | pleia2: hey; it's great you want to step up into that, but I think you need significantly more familiarity with the ops plumbing we have - e.g. be approaching core status in tripleo | 22:30 |
| lifeless | pleia2: it's not a hard requirement, but all the current admins are well down that path with the one exception being derekh - but he's core in toci already which has ~all the same plumbing | 22:31 |
| pleia2 | lifeless: ok | 22:31 |
| *** morazi has quit IRC | 22:32 | |
| pleia2 | dprince and derekh suggested it on the call today so we could get some stuff set up, but they also have other systems we can collaborate on | 22:32 |
| pleia2 | I follow up with them tomorrow | 22:33 |
| lifeless | pleia2: so, from a devops perspective | 22:33 |
| *** rpodolyaka1 has quit IRC | 22:33 | |
| lifeless | pleia2: I want to see all the bring be automated to the extent that a contributor can say 'can a tripleo-cd admin redeploy the test environment heat stack on the undercloud' and have that DTRT | 22:34 |
| * pleia2 nods | 22:34 | |
| lifeless | pleia2: e.g. you should be able to test locally with a virt bm cloud, be confident the images build etc, and point $whoever at the right script in toci to run to build+deploy and be done with it. | 22:34 |
| lifeless | pleia2: I am sure there will be glitches we find, but if there is any manual handholding, we start building up in-person-X-head-state | 22:35 |
| lifeless | which is bad | 22:35 |
| pleia2 | lifeless: sure, we were just trying to work out some of the networking bits that were a bit hard to spec out in the call | 22:35 |
| pleia2 | there are certainly other ways to do that than playing with the real CD cloud :) | 22:35 |
| lifeless | pleia2: ack; I think you should be able to test all those locally with a emulated bm + devstack, or somesuch | 22:36 |
| lifeless | yah | 22:36 |
| openstackgerrit | Clint "SpamapS" Byrum proposed a change to openstack/tripleo-image-elements: Run tests on elements in tox.ini https://review.openstack.org/60056 | 22:38 |
| *** morazi has joined #tripleo | 22:47 | |
| *** cd-undercloud has joined #tripleo | 22:48 | |
| cd-undercloud | ************** overcloud complete status=1 ************ | 22:48 |
| *** cd-undercloud has quit IRC | 22:48 | |
| *** bauzas has quit IRC | 22:55 | |
| *** rongze has joined #tripleo | 22:55 | |
| *** rongze has quit IRC | 23:00 | |
| *** bauzas has joined #tripleo | 23:00 | |
| *** noslzzp has quit IRC | 23:02 | |
| *** noslzzp has joined #tripleo | 23:04 | |
| *** noslzzp has quit IRC | 23:08 | |
| *** ccrouch has quit IRC | 23:10 | |
| *** noslzzp has joined #tripleo | 23:12 | |
| *** cd-undercloud has joined #tripleo | 23:13 | |
| cd-undercloud | ************** overcloud complete status=1 ************ | 23:13 |
| *** cd-undercloud has quit IRC | 23:13 | |
| SpamapS | ERROR: Giving up waiting for overcloud delete to complete after 120 attempts. | 23:14 |
| SpamapS | I wonder if we could get the IRC message to contain the line number of devtest_overcloud.sh where we failed. | 23:14 |
| *** matty_dubs is now known as matty_dubs|gone | 23:18 | |
| *** vipul is now known as vipul-away | 23:21 | |
| SpamapS | lifeless: how can we encode this "move eth2's address to br-ctlplane" thing so we don't have to remember to do it? | 23:22 |
| lifeless | SpamapS: it is encoded | 23:22 |
| lifeless | SpamapS: idempotently even | 23:22 |
| lifeless | SpamapS: but we're not pushing the route from the seed atm, remember ? | 23:23 |
| SpamapS | how so? | 23:23 |
| lifeless | SpamapS: see the ovs agent script | 23:23 |
| lifeless | SpamapS: there is clearly a bug | 23:23 |
| SpamapS | lifeless: we had another mellanox freakout .. :-/ | 23:24 |
| lifeless | SpamapS: oh, you mean with the mellanox workaround? | 23:24 |
| lifeless | SpamapS: NFI | 23:24 |
| lifeless | I just have a command line with ;'s between it | 23:24 |
| SpamapS | like I want to say "make it the way it should be" | 23:24 |
| *** michchap has joined #tripleo | 23:24 | |
| lifeless | yeah, I'm with you on that | 23:24 |
| SpamapS | lifeless: Keep meaning to get back to building an image with the newer version of the driver. | 23:25 |
| openstackgerrit | A change was merged to openstack-infra/tripleo-ci: Drop notcompute element. https://review.openstack.org/59112 | 23:33 |
| openstackgerrit | A change was merged to openstack/tripleo-incubator: Assert undercloud auth before building overcloud. https://review.openstack.org/59237 | 23:34 |
| openstackgerrit | A change was merged to openstack/diskimage-builder: Default name for ramdisks to image. https://review.openstack.org/60046 | 23:37 |
| *** vipul-away is now known as vipul | 23:41 | |
| *** rongze has joined #tripleo | 23:56 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!