| lifeless | tchaypo: -c should be generating a new environment | 00:00 |
|---|---|---|
| tchaypo | dangit. yes, of course i do. | 00:00 |
| *** yamahata__ has quit IRC | 00:09 | |
| openstackgerrit | lifeless proposed a change to openstack/tripleo-image-elements: Fixup Ironic support. https://review.openstack.org/81627 | 00:11 |
| *** CaptTofu has joined #tripleo | 00:13 | |
| *** matsuhashi has joined #tripleo | 00:25 | |
| openstackgerrit | lifeless proposed a change to openstack/tripleo-incubator: Add Ironic support to incubator. https://review.openstack.org/81959 | 00:34 |
| openstackgerrit | lifeless proposed a change to openstack/tripleo-incubator: Add Ironic support to incubator. https://review.openstack.org/81959 | 00:36 |
| *** fandi has quit IRC | 00:49 | |
| *** fandi has joined #tripleo | 00:54 | |
| *** tzumainn has quit IRC | 01:00 | |
| *** tserong has quit IRC | 01:12 | |
| *** BadCub01 has quit IRC | 01:12 | |
| *** tserong has joined #tripleo | 01:12 | |
| *** tserong has joined #tripleo | 01:12 | |
| *** yamahata has joined #tripleo | 01:14 | |
| lifeless | StevenK: around ? error: argument <cpus>: invalid int value: u'PRIVATE' | 01:15 |
| tchaypo | that's definitely an invalid int value | 01:16 |
| * lifeless slowclaps | 01:16 | |
| openstackgerrit | lifeless proposed a change to openstack/tripleo-incubator: Add Ironic support to incubator. https://review.openstack.org/81959 | 01:16 |
| lifeless | ah unquoted ssh_key | 01:18 |
| *** ohadlevy has quit IRC | 01:21 | |
| lifeless | tchaypo: its not a public hol in aus today is it ? | 01:22 |
| tchaypo | If it is, I'm wasting it | 01:22 |
| *** ohadlevy has joined #tripleo | 01:22 | |
| *** CaptTofu has quit IRC | 01:23 | |
| *** eghobo has joined #tripleo | 01:23 | |
| *** ohadlevy is now known as Guest61364 | 01:23 | |
| tchaypo | oh huzzah - only 89 minutes later, check-overcloud-precise finally finished | 01:24 |
| StevenK | lifeless: Blah, I'm fixing patch set 9 and you uploaded 10. | 01:25 |
| StevenK | lifeless: Also, I did the macs for loop differently, and yours has a bug. | 01:25 |
| *** fandi has quit IRC | 01:26 | |
| lifeless | StevenK: I have a further edit sorry :) | 01:26 |
| lifeless | StevenK: I did say I had a fix in the weekend .... so you wouldn't waste time | 01:27 |
| StevenK | lifeless: I didn't see that hilight :-( | 01:27 |
| lifeless | k, I'll send mail too next time | 01:27 |
| lifeless | StevenK: whats the bug in my impl ? | 01:29 |
| StevenK | lifeless: Hint, it's in your seq call | 01:29 |
| StevenK | Oh, that's not a bug. I was reading it like it was my solution. | 01:30 |
| lifeless | have you tested yet? there's a nasty escaping problem | 01:32 |
| StevenK | I am in the middle of it | 01:33 |
| lifeless | nova baremetal-node-create --pm_address=192.168.122.1 --pm_user=robertc '--pm_password="-----BEGIN' RSA PRIVATE 'KEY----- | 01:33 |
| *** nosnos has joined #tripleo | 01:36 | |
| lifeless | StevenK: sorry | 01:37 |
| openstackgerrit | lifeless proposed a change to openstack/tripleo-incubator: Add Ironic support to incubator. https://review.openstack.org/81959 | 01:37 |
| openstackgerrit | lifeless proposed a change to openstack/tripleo-incubator: Remove backwards compatability in setup-baremetal https://review.openstack.org/78467 | 01:37 |
| lifeless | StevenK: I'm debugging the Ironic patch and had to fixup stuff - I don't believe I'll need to push the bottom again and when your patch is up will switch onto it | 01:37 |
| lifeless | StevenK: I humbly suggest you'll want to the escaping patch | 01:37 |
| StevenK | lifeless: Which is still a little broken, no? | 01:40 |
| StevenK | nova baremetal-node-create --pm_address=192.168.122.1 --pm_user=steven '--pm_password=-----BEGIN RSA PRIVATE KEY----- | 01:40 |
| lifeless | StevenK: my patch? no, it works | 01:43 |
| lifeless | StevenK: note the explicit " | 01:43 |
| StevenK | It just looks strange in set -x | 01:45 |
| lifeless | let me eyeball | 01:45 |
| lifeless | see if I rebased it badly | 01:45 |
| lifeless | --pm_password="$(jq -r .pm_password <<< $NODE_JSON)" | 01:46 |
| lifeless | is what that should have | 01:46 |
| openstackgerrit | lifeless proposed a change to openstack/tripleo-incubator: Add Ironic support to incubator. https://review.openstack.org/81959 | 01:48 |
| StevenK | Trying to work out how to get into the DB so I can compare the password to the JSON | 01:49 |
| lifeless | StevenK: mysql nova_bm; | 01:50 |
| * tchaypo -> loonch etc | 01:52 | |
| StevenK | lifeless: Huh. How did this ever work? | 01:54 |
| lifeless | StevenK: we weren't passing the key into nova-bm before | 01:54 |
| StevenK | lifeless: pm_password is a varchar(255) | 01:54 |
| StevenK | So we now have the first 255 characters of the key in the DB | 01:55 |
| lifeless | StevenK: in nova-bm ? Thats fine. | 01:55 |
| lifeless | StevenK: we need it in the json for Ironic | 01:55 |
| lifeless | eventually. | 01:55 |
| StevenK | So it won't get used as is? | 01:56 |
| StevenK | Even though we're setting it in the DB? | 01:56 |
| lifeless | correct | 01:56 |
| lifeless | because nova-bm doesn't read passwords from the db for ssh | 01:57 |
| StevenK | Heh, if tchaypo is going outside for lunch, I hope he bought a boat. | 01:57 |
| StevenK | lifeless: Did you want to +A https://review.openstack.org/#/c/80905/ , or shall we see just how many +1 and +2s it can can collect? :-) | 01:59 |
| tchaypo | it's only lighting raining over here just now | 01:59 |
| tchaypo | s/lighting/lightly/ | 01:59 |
| StevenK | It is bucketing down here | 02:00 |
| tchaypo | I suspect that will change, judging by the radar | 02:00 |
| tchaypo | I was planning on heading out later but might just stay here if it does get heavy | 02:00 |
| StevenK | Yeah. | 02:00 |
| StevenK | The wind and rain are so strong that I was getting rained on from the window that was open about 5cm | 02:01 |
| StevenK | And I'm sitting roughly 3m from the window | 02:01 |
| lifeless | StevenK: it failed jenkins | 02:02 |
| *** eghobo has quit IRC | 02:08 | |
| tchaypo | looking at current radar, it looks like the patch you're in isn't extending east of homebush. There's a smaller patch just east of me, but it looks like if anything is going to hit me it'll be the stuff that's now between wiseman's ferry and gosford moving south | 02:08 |
| *** noslzzp has quit IRC | 02:12 | |
| lifeless | tchaypo: when you traige - e.g. https://bugs.launchpad.net/tripleo/+bug/1292938 please set a priority! - its not triaged without one... | 02:14 |
| uvirtbot | Launchpad bug 1292938 in tripleo "dkms fails due to devtest not requiring linux-headers package as dependency" [Undecided,Triaged] | 02:14 |
| *** CaptTofu has joined #tripleo | 02:15 | |
| tchaypo | lifeless: ack | 02:19 |
| tchaypo | in some cases I've been uncertain about what priority to set but I think I've usually left those ones alone. Not sure what happened in that case. | 02:19 |
| lifeless | web services can have races, probably want to set priority, *then* traige. | 02:21 |
| lifeless | triage. | 02:21 |
| tchaypo | lifeless: https://review.openstack.org/#/c/82405/ is about to pass the final check job | 02:40 |
| tchaypo | (and thus I feel okay asking for your review) | 02:42 |
| tchaypo | I think it avoids the issues with the earlier patch that broke ci | 02:42 |
| tchaypo | okay, I'm getting some decent rain here now, but it's not terribly blowy | 02:44 |
| *** rbrady has quit IRC | 02:51 | |
| *** matsuhashi has quit IRC | 02:55 | |
| *** ramishra has joined #tripleo | 03:02 | |
| *** ramishra_ has joined #tripleo | 03:03 | |
| *** ramishra has quit IRC | 03:07 | |
| *** michchap_ has joined #tripleo | 03:08 | |
| *** michchap has quit IRC | 03:09 | |
| *** CaptTofu has quit IRC | 03:11 | |
| *** CaptTofu has joined #tripleo | 03:12 | |
| StevenK | It's still raining here | 03:13 |
| StevenK | The river has burst its banks | 03:13 |
| *** eghobo has joined #tripleo | 03:13 | |
| StevenK | lifeless: Oh, bleh. | 03:17 |
| *** CaptTofu has quit IRC | 03:17 | |
| StevenK | lifeless: toci_devtest.sh calls setup-baremetal directly, so the overcloud jobs will always fail on the backwards compatibility branch | 03:17 |
| lifeless | StevenK: so, needs to keep compat with command lines | 03:19 |
| lifeless | StevenK: possibly needs a prepatory change to setup-baremetal to have a CLI that is more amenable to keeping compat | 03:20 |
| *** matsuhashi has joined #tripleo | 03:21 | |
| StevenK | Right, so let's resurrect --not-first with $TE_DATAFILE in the remove backwards compatibilty | 03:22 |
| StevenK | Then a toci patch to use --nodes | 03:23 |
| StevenK | And then --not-first can die | 03:23 |
| *** nati_ueno has joined #tripleo | 03:23 | |
| lifeless | actually | 03:24 |
| lifeless | hang on a sec | 03:24 |
| lifeless | we've got the new testenvs now | 03:24 |
| lifeless | we can change toci right now | 03:24 |
| lifeless | it just needs to stop using --first and --not-first entirely. | 03:24 |
| lifeless | no need to change this patch at all | 03:24 |
| StevenK | How? If toci changes to stop using --not-first, current patches will break. | 03:25 |
| lifeless | why? | 03:25 |
| lifeless | current setup-baremetal supports passing a set of nodes in | 03:25 |
| StevenK | It does not | 03:25 |
| lifeless | oh | 03:26 |
| lifeless | it got too neutered | 03:26 |
| StevenK | So, shall I continue with my plan? | 03:26 |
| lifeless | uhm | 03:26 |
| StevenK | Hm | 03:26 |
| lifeless | I don't think your plan as expressed will work | 03:27 |
| lifeless | I think you need a patch to current setup-baremetal to allow someone to pass just nodes in. | 03:27 |
| lifeless | land that. | 03:27 |
| lifeless | Then change toci. | 03:27 |
| lifeless | Then land this patch. | 03:27 |
| lifeless | otherwise we don't have any common-state to do the transition | 03:27 |
| StevenK | My plan was to allow --not-first in the backwards compat branch. After that lands, change toci to use --nodes. And then in a further patch, drop --not-first from setup-baremetal | 03:28 |
| lifeless | you can't change toci to use --nodes until we've *landed* a patch to add --nodes to setup-baremetal. | 03:29 |
| lifeless | -> picking up C. | 03:29 |
| StevenK | lifeless: Which happens in the backwards compat branch | 03:29 |
| StevenK | mordred: If you're still around, what about hotels? | 03:31 |
| lifeless | StevenK: which we can't land until toci is changed | 03:42 |
| lifeless | StevenK: perhaps we're saying the same thing | 03:42 |
| StevenK | lifeless: If the backwards compat branch also supports TE_DATAFILE and --not-first, it supports both uses? | 03:42 |
| lifeless | StevenK: ok, but its not really deleting backwards compat code then is it ? | 03:43 |
| lifeless | :) | 03:43 |
| StevenK | I was hoping you wouldn't notice that. | 03:43 |
| StevenK | But when I finish writing a mail to certain resident of Dallas, I'll look at starting on your plan. | 03:43 |
| lifeless | you rplan is fine too | 03:43 |
| lifeless | I just had cognitive dissonance with the commit message | 03:43 |
| StevenK | Yeah | 03:43 |
| StevenK | lifeless: That toci branch passed jenkins, can has +A? | 03:46 |
| StevenK | https://review.openstack.org/#/c/80905/ that is | 03:47 |
| openstackgerrit | A change was merged to openstack-infra/tripleo-ci: Change how the JSON is updated for readability https://review.openstack.org/80905 | 03:47 |
| StevenK | lifeless: Thanks. | 03:48 |
| *** ekarlso has quit IRC | 03:48 | |
| *** ekarlso has joined #tripleo | 03:48 | |
| StevenK | I think we can drop the fire drill from the topic, too. | 03:48 |
| *** StevenK changes topic to "tripleo-cd running preserve-ephemeral WIP patches and https://review.openstack.org/#/c/62042/ | Using OpenStack to deploy OpenStack;meetings Tuesday 1900 UTC in #openstack-meeting-alt" | 03:49 | |
| openstackgerrit | A change was merged to openstack/tripleo-incubator: Sets disks in devtest run to sda rather than vda. https://review.openstack.org/81764 | 03:51 |
| *** killer_prince has joined #tripleo | 03:52 | |
| *** lazy_prince has quit IRC | 03:53 | |
| lifeless | ok ironic undercloud is blocked on fixing the race condition. | 04:15 |
| *** killer_prince has quit IRC | 04:16 | |
| *** killer_prince has joined #tripleo | 04:18 | |
| *** fandi has joined #tripleo | 04:20 | |
| lifeless | devananda: btw - https://bugs.launchpad.net/ironic/+bug/1295503 - can't deploy ironic using heat today - I've put more detail in the bug | 04:22 |
| uvirtbot | Launchpad bug 1295503 in ironic "ironic nova driver blocks nova-compute startup when ironic isn't available" [Medium,Incomplete] | 04:22 |
| openstackgerrit | lifeless proposed a change to openstack/tripleo-heat-templates: Fix Ironic templates https://review.openstack.org/82414 | 04:24 |
| openstackgerrit | lifeless proposed a change to openstack/tripleo-incubator: Add Ironic support to incubator. https://review.openstack.org/81959 | 04:24 |
| tchaypo | googling for | 04:35 |
| tchaypo | "ip netns tcpdump" mostly shows posts on openstack mailing lists where people are troubleshooting why dhcp requests don't make it onto the interface where dnsmasq is listening | 04:36 |
| tchaypo | either google is spookily knowledgeable about exactly why I'm searching, or it's the only reason people have ever had to run tcpdump under netns | 04:36 |
| StevenK | It's Google. It could be either. | 04:37 |
| *** ramishra_ has quit IRC | 04:39 | |
| *** vkozhukalov has joined #tripleo | 04:43 | |
| *** vkozhukalov has left #tripleo | 04:45 | |
| lifeless | or both | 04:45 |
| *** mestery has joined #tripleo | 04:46 | |
| *** nati_ueno has quit IRC | 04:46 | |
| *** nati_ueno has joined #tripleo | 04:47 | |
| *** tzumainn has joined #tripleo | 04:51 | |
| *** nati_ueno has quit IRC | 04:51 | |
| *** nati_ueno has joined #tripleo | 04:52 | |
| *** killer_prince2 has joined #tripleo | 04:52 | |
| *** killer_prince2 is now known as lazy_prince | 04:52 | |
| *** tzumainn has quit IRC | 04:57 | |
| *** ramishra has joined #tripleo | 05:11 | |
| *** CaptTofu has joined #tripleo | 05:12 | |
| *** mestery has quit IRC | 05:14 | |
| *** nosnos_ has joined #tripleo | 05:16 | |
| *** CaptTofu has quit IRC | 05:17 | |
| openstackgerrit | Om Kumar proposed a change to openstack/tripleo-image-elements: Adds local storage Boot support in case PXE boot fails. https://review.openstack.org/79289 | 05:17 |
| *** nosnos has quit IRC | 05:19 | |
| openstackgerrit | Om Kumar proposed a change to openstack/tripleo-image-elements: Adds local storage Boot support in case PXE boot fails. https://review.openstack.org/79289 | 05:20 |
| *** sdake_ has quit IRC | 05:47 | |
| *** matsuhas_ has joined #tripleo | 06:00 | |
| *** matsuhashi has quit IRC | 06:01 | |
| openstackgerrit | Steve Kowalik proposed a change to openstack/tripleo-incubator: Support for specifying nodes for setup-baremetal https://review.openstack.org/82431 | 06:12 |
| *** mestery has joined #tripleo | 06:13 | |
| *** mestery has quit IRC | 06:13 | |
| *** mestery has joined #tripleo | 06:14 | |
| *** nati_ueno has quit IRC | 06:14 | |
| *** mestery_ has joined #tripleo | 06:15 | |
| *** mestery has quit IRC | 06:15 | |
| *** nosnos has joined #tripleo | 06:19 | |
| *** mestery_ has quit IRC | 06:20 | |
| *** nosnos_ has quit IRC | 06:22 | |
| *** matsuhas_ has quit IRC | 06:31 | |
| *** matsuhashi has joined #tripleo | 06:31 | |
| *** Guest61364 is now known as ohadlevy | 06:49 | |
| *** ohadlevy has joined #tripleo | 06:49 | |
| *** rdopieralski has joined #tripleo | 06:53 | |
| *** jprovazn has joined #tripleo | 07:04 | |
| *** CaptTofu has joined #tripleo | 07:13 | |
| *** michchap has joined #tripleo | 07:18 | |
| *** CaptTofu has quit IRC | 07:18 | |
| *** rdopieralski has quit IRC | 07:19 | |
| *** michchap_ has quit IRC | 07:19 | |
| *** rdopieralski has joined #tripleo | 07:19 | |
| *** ramishra has quit IRC | 07:24 | |
| *** eghobo has quit IRC | 07:32 | |
| *** jistr has joined #tripleo | 07:38 | |
| *** e0ne has joined #tripleo | 07:44 | |
| *** e0ne has quit IRC | 07:47 | |
| openstackgerrit | Jan Provaznik proposed a change to openstack/diskimage-builder: Adds "element-provides" support to element dependencies https://review.openstack.org/79421 | 08:03 |
| *** tzumainn has joined #tripleo | 08:05 | |
| lifeless | Ng: GheRivero: morning | 08:05 |
| *** jtomasek has joined #tripleo | 08:05 | |
| Ng | lifeless: Morning | 08:06 |
| Ng | Late start for me today, taking JJ to an allergy appt at le hospital. | 08:06 |
| Ng | Still on for Reviews Monday though :) | 08:07 |
| lifeless | Ng: ok; I have a couple of bugs from HP that really want investigation as much as reviews need reviewing | 08:07 |
| *** rlandy has joined #tripleo | 08:11 | |
| lifeless | bug 1295732 in particular and bug 1295731 | 08:11 |
| uvirtbot | Launchpad bug 1295732 in tripleo "tripleo devtest overcloud startup takes too long " [High,Triaged] https://launchpad.net/bugs/1295732 | 08:11 |
| uvirtbot | Launchpad bug 1295731 in tripleo "tripleo devtest overcloud image creation takes too long" [High,Triaged] https://launchpad.net/bugs/1295731 | 08:11 |
| Ng | lifeless: Ok. I hope I'll be back in a couple of hours, can have a look then | 08:13 |
| *** mrunge has joined #tripleo | 08:14 | |
| lifeless | Ng: I'll leave some notes high level and a suggested approach | 08:16 |
| *** ramishra has joined #tripleo | 08:21 | |
| *** athomas has joined #tripleo | 08:21 | |
| lsmola_ | SpamapS: hello, are you around? :-) | 08:23 |
| *** bogdando has joined #tripleo | 08:26 | |
| *** bauzas has joined #tripleo | 08:27 | |
| *** hashar has joined #tripleo | 08:28 | |
| *** e0ne has joined #tripleo | 08:30 | |
| *** ifarkas has joined #tripleo | 08:35 | |
| *** ramishra has quit IRC | 08:36 | |
| rcarrillocruz | lifeless: hi, regarding Ubuntu 12.04, saw another bug stating it was not supported on devtest, but can't see any indication of that fact in the dcoumentation. Also, in the install-dependencies we explicitly check for Ubuntu 12.04 and we add a cloud archive repo if so. Should I two bugs, one for fixing documentation about not supporting 12.04 and another one to remove the 12.04 check from install-dependencies? | 08:42 |
| *** pblaho has joined #tripleo | 08:43 | |
| rcarrillocruz | s/Should I/Should I file | 08:43 |
| *** hashar has quit IRC | 08:46 | |
| lifeless | rcarrillocruz: we should document that 12.04 isn't known to work and isn't tested. I've no objection to folk adding support for it (tastefully!) | 08:46 |
| rcarrillocruz | k, opening bug and assigning to self, thx | 08:47 |
| lifeless | rcarrillocruz: so I wouldn't remove the 12.04 check (it was from someones previous attempt to make 12.04 work | 08:47 |
| rcarrillocruz | hmm, k, then i'll add the 'fix' for the jq package and will put in the README that 12.04 isn't known to work | 08:49 |
| *** panda has joined #tripleo | 08:51 | |
| *** lucasagomes has joined #tripleo | 08:52 | |
| GheRivero | morning all | 08:57 |
| *** derekh has joined #tripleo | 08:57 | |
| *** jistr has quit IRC | 08:59 | |
| *** hashar has joined #tripleo | 08:59 | |
| lifeless | GheRivero: hey! see above for some key HP folk issues | 09:00 |
| *** martyntaylor has joined #tripleo | 09:01 | |
| GheRivero | lifeless: looking at it | 09:01 |
| rpodolyaka | morning all | 09:01 |
| *** CaptTofu has joined #tripleo | 09:14 | |
| jprovazn | devtest.sh completed. | 09:15 |
| jprovazn | I love this message | 09:15 |
| *** jistr has joined #tripleo | 09:16 | |
| derekh | lifeless: I've prepared the commit to add the rh rack as a provider for tripleo tests, but I'd like to reduce the nodepoll VM's to 8G first if we can any thoughts on merging this https://review.openstack.org/#/c/79362/ so we can move back to 8G VM's ? I'll recheck it now to get some more time comparisons | 09:17 |
| *** CaptTofu has quit IRC | 09:19 | |
| *** martyntaylor has left #tripleo | 09:20 | |
| *** pbelanyi has joined #tripleo | 09:24 | |
| *** ccrouch has joined #tripleo | 09:26 | |
| *** e0ne_ has joined #tripleo | 09:27 | |
| *** e0ne has quit IRC | 09:29 | |
| lifeless | derekh: I think there's another full copy thats snuck in somehow too | 09:29 |
| lifeless | derekh: I'm very worried about moving the raw image to disk, because we'll pay fsyncs | 09:30 |
| derekh | lifeless: another full copy of the image in the tmpfs? Ya I get the worry but I don't see any way around it as is, I'll see if I can poke at it a bit more today to see if I can figure out another solution | 09:32 |
| lifeless | derekh: no, on local disk | 09:35 |
| lifeless | derekh: just a suspicion but I haven't dug into it | 09:35 |
| derekh | lifeless: ok, will take a look | 09:36 |
| lifeless | derekh: there was also a patch to look for more memory (specifically 2* MIN_IMAGE_SIZE) or something | 09:37 |
| lifeless | what else can we do | 09:37 |
| lifeless | we could delete the git trees | 09:38 |
| lifeless | and purge packages that we don't need | 09:38 |
| lifeless | and dan's work to consolidate venvs | 09:38 |
| lifeless | which has caveats but we agreed to give it a go | 09:38 |
| openstackgerrit | Jan Provaznik proposed a change to openstack/diskimage-builder: Adds libmariadb-dev package mapping https://review.openstack.org/79874 | 09:40 |
| derekh | lifeless: this one that SpamapS did https://review.openstack.org/#/c/79074/ this prvents us from using more ram on bigger hosts so we know the size of the tmpfs and don't blindly use RAM/2, | 09:40 |
| derekh | lifeless: ya deleting the git trees and using a single venv should bring us below the limit | 09:41 |
| derekh | and speed other stuff up also | 09:41 |
| derekh | so how about I add something to source-repositories that causes it to remove git trees at the and of a install, any element that need the git tree (If we have any) should take a copy before its deleted (or mark it as donotdelete) | 09:43 |
| derekh | lifeless: ^ | 09:43 |
| lifeless | derekh: that sounds like a good starting point | 09:46 |
| lifeless | 300M or so | 09:47 |
| lifeless | 900M in the venvs | 09:47 |
| derekh | lifeless: ok, test to see what happens when git trees are removed in finalise.d is running, lets see what happens | 09:49 |
| *** ccrouch has quit IRC | 09:49 | |
| *** vkozhukalov has joined #tripleo | 09:50 | |
| lifeless | night all | 09:51 |
| *** howleyt has quit IRC | 09:51 | |
| openstackgerrit | Derek Higgins proposed a change to openstack-infra/tripleo-ci: Add hostname of Test env to json https://review.openstack.org/80603 | 09:53 |
| ProfFalken | is there a framework for testing the functionality of Tripleo elements? | 09:57 |
| ProfFalken | Devtest confirms that they install correctly, but where do I write the codeto make sure they have configured as expected if they are elements that cannot be tested via a VM Lifecycle check? | 09:58 |
| *** akrivoka has joined #tripleo | 09:59 | |
| GheRivero | I'm not aware of such option yet | 09:59 |
| openstackgerrit | Derek Higgins proposed a change to openstack-infra/tripleo-ci: Remove "Revert https://review.openstack.org/#/c/58533/19" https://review.openstack.org/82462 | 10:09 |
| *** vkozhukalov has left #tripleo | 10:11 | |
| openstackgerrit | Ricardo Carrillo Cruz proposed a change to openstack/tripleo-incubator: Add backports repo for jq package https://review.openstack.org/82463 | 10:19 |
| *** jistr has quit IRC | 10:24 | |
| *** jistr has joined #tripleo | 10:27 | |
| openstackgerrit | A change was merged to openstack/tripleo-image-elements: Don't install percona-toolkit if distro is RHEL7 https://review.openstack.org/78032 | 10:28 |
| *** jp_at_hp has joined #tripleo | 10:28 | |
| *** matsuhashi has quit IRC | 10:33 | |
| *** noslzzp has joined #tripleo | 10:34 | |
| *** matsuhas_ has joined #tripleo | 10:35 | |
| *** yamahata has quit IRC | 10:43 | |
| *** CaptTofu has joined #tripleo | 10:48 | |
| *** akrivoka has quit IRC | 10:48 | |
| *** fandi has quit IRC | 10:49 | |
| *** hewbrocca has joined #tripleo | 10:50 | |
| *** vkozhukalov has joined #tripleo | 10:54 | |
| ProfFalken | GheRivero: ok, fair enough, I might have a think about how best to do that if I get a chance... | 11:04 |
| *** matsuhas_ has quit IRC | 11:07 | |
| *** athomas has quit IRC | 11:16 | |
| *** nosnos has quit IRC | 11:23 | |
| *** athomas has joined #tripleo | 11:24 | |
| *** akrivoka has joined #tripleo | 11:27 | |
| openstackgerrit | Ladislav Smola proposed a change to openstack/tripleo-heat-templates: WIP heat stack-update quick fix https://review.openstack.org/82479 | 11:27 |
| rcarrillocruz | i need to spend some time installing proxies at home, running devtest on a 3MB DSL is not very productive :/ | 11:27 |
| GheRivero | 3MB DSL? Sure you need proxies and mirror! | 11:37 |
| rcarrillocruz | heh, yeah, i read your blog post, it indeed inspired me this thought ;-) | 11:42 |
| openstackgerrit | A change was merged to openstack/tripleo-incubator: Ensure which is installed on Red Hat https://review.openstack.org/81836 | 11:45 |
| *** weshay has joined #tripleo | 11:46 | |
| GheRivero | another one will be coming about how to configure the mirrors | 11:48 |
| rcarrillocruz | cool | 11:53 |
| *** matsuhashi has joined #tripleo | 11:53 | |
| *** nosnos has joined #tripleo | 11:54 | |
| openstackgerrit | Jenkins proposed a change to openstack/os-collect-config: Updated from global requirements https://review.openstack.org/81648 | 11:54 |
| *** lucasagomes is now known as lucas-hungry | 12:05 | |
| lazy_prince | review pls. https://review.openstack.org/#/c/79873/ (Refactor code to select boot kernel). It has passed ci and is waiting for review. | 12:07 |
| *** hewbrocca has quit IRC | 12:08 | |
| openstackgerrit | therese-mchale proposed a change to openstack/tripleo-image-elements: Secure the mysql installation https://review.openstack.org/80570 | 12:08 |
| *** matsuhashi has quit IRC | 12:14 | |
| *** matsuhas_ has joined #tripleo | 12:17 | |
| *** tserong has quit IRC | 12:18 | |
| *** hewbrocca has joined #tripleo | 12:19 | |
| *** tserong has joined #tripleo | 12:19 | |
| *** tserong has joined #tripleo | 12:19 | |
| openstackgerrit | Derek Higgins proposed a change to openstack/tripleo-incubator: Set selinux default to permissive on Fedora https://review.openstack.org/81826 | 12:22 |
| *** e0ne_ has quit IRC | 12:24 | |
| *** lazy_prince has quit IRC | 12:31 | |
| *** rbrady has joined #tripleo | 12:33 | |
| *** matty_dubs|brno is now known as matty_dubs | 12:46 | |
| *** vkozhukalov has quit IRC | 12:49 | |
| *** vkozhukalov1 has joined #tripleo | 12:50 | |
| *** e0ne has joined #tripleo | 12:55 | |
| *** e0ne has quit IRC | 12:59 | |
| *** dprince has joined #tripleo | 13:01 | |
| *** tserong has quit IRC | 13:06 | |
| *** lucas-hungry is now known as lucasagomes | 13:07 | |
| *** tserong has joined #tripleo | 13:09 | |
| *** lblanchard has joined #tripleo | 13:10 | |
| *** e0ne has joined #tripleo | 13:11 | |
| *** matsuhas_ has quit IRC | 13:12 | |
| *** matsuhashi has joined #tripleo | 13:13 | |
| *** matsuhashi has quit IRC | 13:17 | |
| *** julim has joined #tripleo | 13:18 | |
| openstackgerrit | Dan Prince proposed a change to openstack/diskimage-builder: baremetal: correct the path of ifcfg-eth0 https://review.openstack.org/82093 | 13:22 |
| *** ccrouch has joined #tripleo | 13:22 | |
| openstackgerrit | Ladislav Smola proposed a change to openstack/tuskar: Adding swift role back https://review.openstack.org/82499 | 13:23 |
| *** CaptTofu has quit IRC | 13:24 | |
| *** nosnos has quit IRC | 13:24 | |
| *** nosnos has joined #tripleo | 13:25 | |
| *** nosnos has quit IRC | 13:29 | |
| *** geerdest has joined #tripleo | 13:34 | |
| *** geerdest has quit IRC | 13:34 | |
| *** geerdest has joined #tripleo | 13:34 | |
| *** andreaf has joined #tripleo | 13:36 | |
| geerdest | jprovazn: how do i pull in the file you mention in https://review.openstack.org/#/c/79951/ | 13:37 |
| jprovazn | geerdest, hi, actually I messed path - it's in diskimage-builder repo - the path is: diskimage-builder/elements/redhat-common/bin/map-packages | 13:40 |
| *** gcha has joined #tripleo | 13:42 | |
| *** jrist has joined #tripleo | 13:44 | |
| *** newell has joined #tripleo | 13:44 | |
| geerdest | jprovazn: so should i just drop the file from my review then? | 13:47 |
| jprovazn | geerdest, yes - I think that any additional new deps should be added to the diskimage-builder/elements/redhat-common/bin/map-packages instead of adding the file | 13:49 |
| openstackgerrit | Trent Geerdes proposed a change to openstack/tripleo-image-elements: Adding more .deb dependencies for pypi-mirror on 13.10 https://review.openstack.org/79951 | 13:51 |
| *** yamahata has joined #tripleo | 13:54 | |
| openstackgerrit | Dan Prince proposed a change to openstack/tripleo-image-elements: TEST ONLY: make nova depend on common-venv https://review.openstack.org/79989 | 13:57 |
| openstackgerrit | Dan Prince proposed a change to openstack/tripleo-image-elements: Openstack-clients: don't hard code venv https://review.openstack.org/79988 | 13:57 |
| openstackgerrit | Dan Prince proposed a change to openstack/tripleo-image-elements: Update elements to use os-svc-daemon -i. https://review.openstack.org/76963 | 13:57 |
| openstackgerrit | Dan Prince proposed a change to openstack/tripleo-image-elements: Update elements to use os-svc-install -i. https://review.openstack.org/76964 | 13:57 |
| openstackgerrit | Dan Prince proposed a change to openstack/tripleo-image-elements: Wire in OPENSTACK_VENV_DIR default if set https://review.openstack.org/76965 | 13:57 |
| openstackgerrit | Dan Prince proposed a change to openstack/tripleo-image-elements: Wire in _EXTRA_INSTALL_OPTS... https://review.openstack.org/76966 | 13:57 |
| openstackgerrit | Dan Prince proposed a change to openstack/tripleo-image-elements: Add a new common-venv element https://review.openstack.org/76967 | 13:57 |
| openstackgerrit | Dan Prince proposed a change to openstack/tripleo-image-elements: Add -i options to os-svc-install/os-svc-daemon... https://review.openstack.org/76952 | 13:57 |
| *** vkozhukalov1 has left #tripleo | 13:57 | |
| ProfFalken | wow! How fast can you type "git review" and hit enter?! | 13:57 |
| *** nosnos has joined #tripleo | 13:58 | |
| *** matsuhashi has joined #tripleo | 13:59 | |
| *** nosnos has quit IRC | 14:00 | |
| *** ccrouch has left #tripleo | 14:01 | |
| *** nosnos has joined #tripleo | 14:01 | |
| openstackgerrit | Dan Prince proposed a change to openstack/tripleo-image-elements: TEST ONLY: make nova depend on common-venv https://review.openstack.org/79989 | 14:03 |
| openstackgerrit | Dan Prince proposed a change to openstack/tripleo-image-elements: Add a new common-venv element https://review.openstack.org/76967 | 14:03 |
| *** nosnos has quit IRC | 14:05 | |
| *** rlandy_ has joined #tripleo | 14:08 | |
| openstackgerrit | Matthew Macdonald-Wallace proposed a change to openstack/tripleo-image-elements: Install the check_mk Agent https://review.openstack.org/81485 | 14:10 |
| *** jp_at_hp has quit IRC | 14:10 | |
| *** CaptTofu has joined #tripleo | 14:10 | |
| *** nosnos has joined #tripleo | 14:11 | |
| *** rlandy has quit IRC | 14:11 | |
| jprovazn | geerdest, to be sure - complementary patch for https://review.openstack.org/79951 which adds package mapping to diskimage-builder is on the way? | 14:13 |
| *** vkozhukalov1 has joined #tripleo | 14:13 | |
| *** vkozhukalov1 has left #tripleo | 14:14 | |
| openstackgerrit | therese-mchale proposed a change to openstack/tripleo-image-elements: Add a softlink for /var/log/mysql https://review.openstack.org/80567 | 14:14 |
| *** vkozhukalov1 has joined #tripleo | 14:14 | |
| *** nosnos has quit IRC | 14:14 | |
| geerdest | jprovazn: yes up shortly | 14:15 |
| *** vkozhukalov1 has left #tripleo | 14:16 | |
| *** matsuhashi has quit IRC | 14:16 | |
| *** jp_at_hp has joined #tripleo | 14:16 | |
| *** rwsu has joined #tripleo | 14:17 | |
| *** ramishra has joined #tripleo | 14:21 | |
| GheRivero | ProfFalken: dependent patches are uploaded all at the same time | 14:24 |
| openstackgerrit | Trent Geerdes proposed a change to openstack/diskimage-builder: Adding pypi-mirror dependencies for redhat https://review.openstack.org/82513 | 14:24 |
| ProfFalken | GheRivero: heh, yeah, sorry, failed attempt at humour... :) | 14:25 |
| GheRivero | ProfFalken: my bad. Not too good with ironies :) | 14:26 |
| andreaf | hi - I'm trying to get the tempest element back to working state - see https://review.openstack.org/#/c/81813/ and https://review.openstack.org/#/c/82035/ - anyone for a review? | 14:28 |
| *** vkozhukalov1 has joined #tripleo | 14:29 | |
| *** matsuhashi has joined #tripleo | 14:32 | |
| jprovazn | geerdest, cool | 14:32 |
| *** lucasagomes_ has joined #tripleo | 14:33 | |
| *** lucasagomes has quit IRC | 14:34 | |
| *** DuncanT- has quit IRC | 14:36 | |
| *** lucasagomes_ is now known as lucasagomes | 14:37 | |
| openstackgerrit | Jan Provaznik proposed a change to openstack/tripleo-image-elements: Adds mysql-dev and mariadb-dev elements https://review.openstack.org/79886 | 14:48 |
| openstackgerrit | Jan Provaznik proposed a change to openstack/tripleo-image-elements: Adds mariadb element https://review.openstack.org/79884 | 14:48 |
| openstackgerrit | Jan Provaznik proposed a change to openstack/tripleo-image-elements: Adds mysql-common element https://review.openstack.org/79885 | 14:48 |
| openstackgerrit | A change was merged to openstack/tripleo-image-elements: Add missing +x https://review.openstack.org/82387 | 14:50 |
| *** bauzas has quit IRC | 14:50 | |
| SpamapS | lsmola_: I am here now. | 14:53 |
| lsmola_ | SpamapS: hello | 14:55 |
| SpamapS | derekh: regarding removing git trees in source-repositories.. I have a patch that does that, but several folk didn't like the way I specified "cleanup" in the *-source-repositories files. | 14:55 |
| SpamapS | derekh: so the plan was to switch it to yaml | 14:56 |
| SpamapS | I wonder if there is a jq but for yaml | 14:56 |
| lsmola_ | SpamapS: regarding the https://bugs.launchpad.net/tripleo/+bug/1291905 | 14:56 |
| uvirtbot | Launchpad bug 1291905 in tripleo "heat stack-update fails on timeout" [High,Triaged] | 14:56 |
| SpamapS | would be rather nice if there were | 14:56 |
| SpamapS | lsmola_: yes | 14:56 |
| geerdest | i had a change merged a couple weeks back but it had the wrong author/email for me. any way to fix that up? | 14:56 |
| lsmola_ | SpamapS: seems like the easiest option might be to define one COmpletionCondition and Handle for e.g. Compute | 14:57 |
| lsmola_ | SpamapS: and set count attribute on the completionCondition, same as number of Computes | 14:57 |
| lsmola_ | SpamapS: and pass somehow Unique ID to that call | 14:58 |
| derekh | SpamapS: ahh ok, I remember no that you mentioned it, this is what I am testing at the moment, initial test suggest it works except fro the openstack-clients http://paste.openstack.org/show/74148/ | 14:58 |
| SpamapS | lsmola_: hm | 14:58 |
| derekh | SpamapS: but if your working on something diferent I'll abandon this | 14:59 |
| SpamapS | derekh: I like the idea of "we now just delete them all.. deal with it" .. but it would be better if we can ease element users into it. | 14:59 |
| lsmola_ | SpamapS: or did you have something easier in mind? I don't really understand how zaneb wanted to have it solved | 14:59 |
| *** bauzas has joined #tripleo | 14:59 | |
| SpamapS | derekh: No mine was abandoned a month ago because I never found the time to convert things to yaml. | 14:59 |
| SpamapS | lsmola_: well I don't think ZaneB realised you are already using UpdateWaitConditionHandle :) | 14:59 |
| SpamapS | lsmola_: if you're seeing auth problems with _up to date_ wait condition handles that is a bug in Heat. | 15:00 |
| lsmola_ | SpamapS: I think it was used only for not compute, but that itself doesn't solve the issue of missing resource | 15:00 |
| lsmola_ | SpamapS: no, the auth problem is not the issue :-) | 15:01 |
| SpamapS | ahh Ok just getting through the comments | 15:01 |
| * SpamapS will read more and talk less for a moment | 15:01 | |
| lsmola_ | SpamapS: the problem is described little lower, if you create a new resource and then stack-update, it can't receive completed signal, cause the CompletionHandle resource is not accessible | 15:02 |
| derekh | SpamapS: The only elements which I found that had a problem with deleting all git repo's was openstack-clients which can be fixed up, but then again that only worries about whats in devtest | 15:02 |
| SpamapS | lsmola_: why isn't CompletionHandle accessible? It should be created well before the server. | 15:03 |
| *** vkozhukalov1 has left #tripleo | 15:03 | |
| lsmola_ | SpamapS: problem is that heat always reaches the resource through template, but for the CompletionHandle, it uses the current_template which doesn't contain the resource yet | 15:03 |
| SpamapS | derekh: we are not the only users of diskimage-builder | 15:03 |
| SpamapS | derekh: breaking interfaces is not something we can do lightly. | 15:03 |
| derekh | SpamapS: yup, thats what I was getting at when I said " but then again that only worries about whats in devtest" | 15:04 |
| lsmola_ | SpamapS: the resource itself is in the database, it's just how it is being initialized, they still have some metadata only in the template, that is why it needs the template | 15:04 |
| SpamapS | derekh: my thinking was to switch to yaml, add a 'cleanup: true' to our git trees, and start issuing deprecation warnings that use of the non-yaml source-repositories is deprecated, and then if cleanup isn't specified in the yaml also issue a deprecation warning that the default will be switched to true | 15:04 |
| zigo | lifeless: FYI, I have uploaded a new version of testresources in Debian, with support for Python3. I've put the package under PKG OpenStack team, let me know if you don't want this, in which case I can revert. | 15:05 |
| SpamapS | derekh: I wouldn't mind picking that back up if it helps you get RH region up sooner and you have other things you need to focus on. | 15:05 |
| zigo | (and I know you're probably sleeping ... :) ) | 15:05 |
| SpamapS | zigo: I'm almost 100% certain.. mabye 99.999% .. that lifeless would like the latest version of testresources everywhere that testresources is consumed :) | 15:06 |
| zigo | SpamapS: The latest version was already in Debian, but with no Python3 package. And I'm slowly converting EVERYTHING to python3 ... :) | 15:07 |
| derekh | SpamapS: it would help us get up and running quicker (or more accurately It would allow us to get up and running with more capacity. we're pretty much ready now), I can do it either which ever you prefer | 15:07 |
| zigo | SpamapS: Oh, and of course, guys in ubuntu didn't forward their changes, as always ... :/ | 15:07 |
| lsmola_ | SpamapS: this is example of how to call one handle from shardy, http://paste.fedoraproject.org/88017/60519139/ | 15:07 |
| SpamapS | lsmola_: not sure I understand. Is the resource still in 'INIT_COMPLETE'? The graph has a very clear dependency from server -> handle .. so the handle must be CREATE_COMPLETE or UPDATE_COMPLETE before the server will be updated. | 15:07 |
| lsmola_ | SpamapS: not very nice syntax, but it should work | 15:07 |
| *** vkozhukalov has joined #tripleo | 15:08 | |
| zigo | James page does, the others don't. | 15:08 |
| SpamapS | derekh: do you want to look at reducing the venv footprint? I've been wondering if we can't just use something that de-dupes files. | 15:08 |
| *** mrunge has quit IRC | 15:08 | |
| lsmola_ | SpamapS: so during the update, you will not see e.g. NovaCompute1Handle in resource list. if it is the one you are creating | 15:08 |
| zigo | SpamapS: lsmola_: Have you seen that tuskar is already in Experimental? :) | 15:09 |
| zigo | https://packages.debian.org/experimental/tuskar | 15:09 |
| lsmola_ | SpamapS: when you sent cfn-signal from the resource that the resource is completed | 15:09 |
| derekh | SpamapS: dprince has been doing that if that lands it will probably get us passed the same problem https://review.openstack.org/#/q/status:open+project:openstack/tripleo-image-elements+branch:master+topic:common_venv,n,z | 15:09 |
| SpamapS | zigo: Nice. That is the appropriate place for it given that it isn't really a working thing. Well done getting it out there and easily consumable for the masses. :) | 15:09 |
| lsmola_ | SpamapS: it sends only stack_id and resource, then it loads the template from stack and the resource through template | 15:10 |
| SpamapS | zigo: are you going to champion the debian package based TripleO the same way slagle has championed the Fedora/RH one? :) | 15:10 |
| lsmola_ | SpamapS: but the resource Compute1 in not in the current template, so it throws an error | 15:10 |
| SpamapS | derekh: that's not the same thing | 15:10 |
| lsmola_ | SpamapS: the Compute1 is only in the updated template | 15:10 |
| SpamapS | derekh: You know I don't support such a move. :) | 15:10 |
| zigo | Only a very small number of TripleO packages are still waiting for NEW queu approval. | 15:11 |
| zigo | Do you guys think TripleO will be ready and working for Icehouse in a few weeks? | 15:11 |
| SpamapS | zigo: no. | 15:11 |
| zigo | :/ | 15:11 |
| SpamapS | zigo: RH guys may disagree. For my money, releases are a waste of time. ;) | 15:11 |
| slagle | for some definitions of "working" :) | 15:12 |
| SpamapS | Well yeah, it works now.. You could build a pretty wicked small non-HA cloud right now with it. | 15:12 |
| *** vkozhukalov has quit IRC | 15:12 | |
| slagle | right | 15:12 |
| lsmola_ | SpamapS: so what helps is using only one CompletionHandle resource, so it is always visible :-) | 15:13 |
| SpamapS | But.. updates will be best-effort.. we're not quite testing those yet. :-P | 15:13 |
| slagle | and we aim to have a working icehouse deployment using the tripleo stable branches | 15:13 |
| SpamapS | lsmola_: OH! | 15:13 |
| SpamapS | lsmola_: that is a DEFINITE bug | 15:13 |
| slagle | so you can deploy openstack icehouse using the icehouse branches of the tripleo repos | 15:13 |
| SpamapS | lsmola_: -> #heat | 15:13 |
| SpamapS | slagle: working, just maybe not as featureful as we'd hope? :) | 15:14 |
| lsmola_ | SpamapS: yep, though we are looking for some fix for icehouse | 15:14 |
| derekh | SpamapS: ahh ok, looks like the 2 approaches are heading for a collision so.... | 15:14 |
| *** eghobo has joined #tripleo | 15:16 | |
| *** beekneemech is now known as bnemec | 15:20 | |
| openstackgerrit | A change was merged to openstack/tripleo-image-elements: Adding more .deb dependencies for pypi-mirror on 13.10 https://review.openstack.org/79951 | 15:23 |
| *** matsuhashi has quit IRC | 15:25 | |
| *** nati_ueno has joined #tripleo | 15:26 | |
| *** matsuhas_ has joined #tripleo | 15:28 | |
| SpamapS | derekh: Well I hope we can just make the venv build time fast with wheels. | 15:30 |
| SpamapS | derekh: in my testing it is in fact an order of magnitude faster to use wheels that are already built for the platform you're on. | 15:30 |
| SpamapS | derekh: but I ran into i386/x86_64 issues that would have to be resolved. | 15:31 |
| *** matsuhas_ has quit IRC | 15:33 | |
| *** jpeeler has joined #tripleo | 15:33 | |
| derekh | SpamapS: ok, I havn't really looked into wheels much at all, will take a look | 15:33 |
| SpamapS | derekh: but having separate venvs is really quite important for the CD use case. | 15:34 |
| SpamapS | and we don't want our CI to skip either of the two chief use cases (CD and shipping packages) | 15:34 |
| SpamapS | oops.. we do skip packages ;) | 15:34 |
| *** CaptTofu has quit IRC | 15:39 | |
| derekh | SpamapS: agreed whatever we CI has to be our upstream use case, why are separate venvs really important for CD ? so we can upgrade a single service without effecting others? wouldn't CI fully testing images cover us there regardless of which approach we used? | 15:39 |
| SpamapS | derekh: because sometimes the end user _must_ diverge dependencies. | 15:40 |
| *** CaptTofu has joined #tripleo | 15:40 | |
| SpamapS | derekh: and we don't want that need to result in an 800MB bump in image size. ;) | 15:40 |
| openstackgerrit | Jiri Stransky proposed a change to openstack/os-cloud-config: Initial Keystone users, roles, domains, projects https://review.openstack.org/78148 | 15:40 |
| *** andreaf2 has joined #tripleo | 15:44 | |
| derekh | SpamapS: If that case arose (and I accept it may), then we would just have an additional single venv for a particular component, there would be a bump in image size but probably closer to 100M | 15:44 |
| *** andreaf has quit IRC | 15:45 | |
| *** CaptTofu has quit IRC | 15:45 | |
| *** bauzas has quit IRC | 15:49 | |
| openstackgerrit | tom-howley proposed a change to openstack/tripleo-image-elements: WIP - Add keepalive option to rabbitmq config https://review.openstack.org/81920 | 15:50 |
| *** howleyt has joined #tripleo | 15:51 | |
| *** BadCub01 has joined #tripleo | 15:53 | |
| SpamapS | derekh: this isn't "we" | 15:56 |
| SpamapS | derekh: this is "users" | 15:56 |
| SpamapS | derekh: so common-venv has to have not a switch "yes or no" but "this one or that one" | 15:56 |
| SpamapS | derekh: another way to go, IMO, is to actually automatically gather all of the requirements.txts, and try aggressively to create the common-venvs that exist based on the common requirements. | 15:57 |
| SpamapS | but that is also a bit too clever and black-boxy for my sysadmin side :) | 15:58 |
| *** blamar has joined #tripleo | 15:59 | |
| *** blamar has quit IRC | 15:59 | |
| *** blamar has joined #tripleo | 16:00 | |
| openstackgerrit | James Slagle proposed a change to openstack/tripleo-image-elements: Create and use libvirtd group for package install https://review.openstack.org/82529 | 16:00 |
| *** michchap has quit IRC | 16:01 | |
| *** ifarkas has quit IRC | 16:04 | |
| derekh | SpamapS: there is a switch to split out individual venvs, eg.g. export GLANCE_VENV_DIR=/opt/stack/venvs/glance | 16:04 |
| derekh | SpamapS: I did head down the road of looking into the auto gather commons deps idea but gave up when I saw what dprince was doing | 16:05 |
| SpamapS | derekh: ok good then it may be time for me to review that :) | 16:06 |
| *** cwolferh has joined #tripleo | 16:06 | |
| SpamapS | derekh: since it is 7x the savings.. probably worth focusing on helping dprince first :) | 16:06 |
| SpamapS | derekh: there is also another loss that we get which is that we don't detect when a project has missed a dependency | 16:06 |
| SpamapS | derekh: which becomes a time bomb for the case that a service gets split out later | 16:07 |
| derekh | SpamapS: yup, true, periodic jobs to test with multiple venvs ? | 16:07 |
| derekh | brb dorr bell | 16:07 |
| jistr | hi guys, just a quick check - when i have a chicken-and-egg patches involving two OpenStack projects, gerrit cannot help me solve this, can it? I mean patch in project A breaks build, patch in project B breaks build, but if Jenkins tested and merged changes in both projects at the same time, everything would work fine. | 16:09 |
| *** rdopieralski has quit IRC | 16:09 | |
| SpamapS | jistr: right Gerrit cannot help you | 16:10 |
| jistr | i have to create a "logic forking" patch in project A, that will work with both states of project B, and when patch to project B gets merged, i can remove the "forking" in project A | 16:10 |
| jistr | correct? | 16:10 |
| SpamapS | jistr: You need to build in backward compatibility into one of the patches. | 16:10 |
| jistr | SpamapS: cool, thx | 16:10 |
| SpamapS | jistr: that is also courteous to deployers, as they may not wait for both commits to try and deploy to production | 16:11 |
| jistr | SpamapS: yeah that makes sense, thanks | 16:11 |
| SpamapS | remember that with CD, there is no "stabilize" phase | 16:11 |
| SpamapS | all commits are stable, or not merged | 16:12 |
| SpamapS | and stable is determined by CI :) | 16:12 |
| *** eghobo has quit IRC | 16:16 | |
| *** michchap has joined #tripleo | 16:18 | |
| devananda | SpamapS: g'morning! | 16:18 |
| devananda | SpamapS: when you have a few minutes, I think we should chat about https://bugs.launchpad.net/ironic/+bug/1295503 | 16:18 |
| uvirtbot | Launchpad bug 1295503 in ironic "ironic nova driver blocks nova-compute startup when ironic isn't available" [Medium,Incomplete] | 16:18 |
| *** sdake_ has joined #tripleo | 16:18 | |
| *** sdake_ has quit IRC | 16:18 | |
| *** sdake_ has joined #tripleo | 16:18 | |
| *** eghobo has joined #tripleo | 16:22 | |
| andreaf2 | derek: ping | 16:24 |
| andreaf2 | derekh: ping | 16:24 |
| derekh | andreaf2: hi | 16:28 |
| andreaf2 | derekh: hi - I'm looking at tripleo-ci - specifically at the tempest element that exists for it | 16:29 |
| andreaf2 | derekh: I proposed a couple of patches for it, but before going forward I wanted to chat with you about it | 16:29 |
| derekh | andreaf2: tripleo-ci doesn't currently run tempest, although lifeless mentions he wanted to add it | 16:30 |
| andreaf2 | derekh: yes indeed, I'd like to contribute to getting working, but I'd like to understand what was the design so far | 16:31 |
| derekh | andreaf2: he also mentioned he wanted to make it more devstack like, I'm not sure if that involved changing that element or doing something else | 16:31 |
| andreaf2 | derekh: ok so I'll check with lifeless | 16:32 |
| andreaf2 | derekh: thanks | 16:33 |
| derekh | andreaf2: the idea with that element was that you could build it in an image then source the credentials and just run run-tempest | 16:33 |
| andreaf2 | derekh: right, so tunning tempest from the overcloud non-compute node or? | 16:34 |
| andreaf2 | s/tunning/running | 16:34 |
| derekh | andreaf2: anything known to not work with the tripleo overcloud was filtered out by tests2skip.txt but I'm not sure if this file it still uptodate, I havn't tried it in a while | 16:34 |
| *** nati_ueno has quit IRC | 16:35 | |
| derekh | andreaf2: when I was testing I used to run it from the overcloud notdompute node, but I think it should work from another non overcloud node aswell | 16:35 |
| derekh | *notcompute | 16:35 |
| *** sff has quit IRC | 16:35 | |
| bnemec | The last time I tried to run tempest against the overcloud I had quite a few problems, but it's possible that was issues on my end. | 16:36 |
| andreaf2 | derekh: if I understand TOCI right, a devstack-like VM from nodepool is used as entry point, to call a script which creates seed, undercloud and overcloud - so we could even run tempest from the initial node? and configure it using testenv.json | 16:38 |
| derekh | andreaf2: Yup, thats correct and should work, we have access to the overcloud API's from the nodepool node. | 16:41 |
| bnemec | We don't have tempest installed on the nodepool node though. | 16:43 |
| derekh | andreaf2: although, an alternative approach would be to include the tempest element in the user image and then run tempest from there, this would allow us to use the tempest element we have, | 16:43 |
| bnemec | You have to be careful with tempest too because if it gets out of sync with the other projects you can get bogus failures. | 16:45 |
| bnemec | There are quite a few cross-dependencies because tempest is used to test new functionality and such. | 16:46 |
| *** hashar has quit IRC | 16:47 | |
| andreaf2 | bnemec: right at the moment the issue is that changes are not gated via tempest on a tripleo env, so it is well possible to change something in either the tests or the services which passes devstack gate but breaks toci | 16:47 |
| bnemec | andreaf2: Right, but we're not in the gate at all right now. | 16:48 |
| derekh | andreaf2: yup, we have this problem for all of the openstack non tripleo projects and get burned regularly | 16:48 |
| bnemec | Honestly, I don't know that we want to start running tempest until we are in the gate. | 16:48 |
| bnemec | The amount of breakage we get already is a serious problem. | 16:49 |
| bnemec | If we also start failing CI every time a change breaks tempest we may never merge anything. :-) | 16:49 |
| derekh | maybe running tempest that would warn but not fail would be a good middle ground for now | 16:49 |
| andreaf2 | bnemec: ok so you mean first get a voting job on all services which runs toci, and then add tempest to it | 16:50 |
| bnemec | andreaf2: Right. | 16:50 |
| *** shakayumi has joined #tripleo | 16:51 | |
| *** pblaho has quit IRC | 16:52 | |
| andreaf2 | derekh, bnemec: I think it would be good to start getting some kind of indication if some tempest at leat are running, something like a the smoke subset. The current check only creates a VM and pings it I think. So we know that we can deploy a cloud, but we don't know much about whether the deployed cloud works | 16:52 |
| andreaf2 | derekh, bnemec: I'm happy to put some effort in bringing this forward in parallel to getting toci in the gate, but I'd like to have an agreement on where the test are going to run and how they're going to be configured first | 16:54 |
| derekh | andreaf2: so we got 2 options for where its going to run, 1) and tempest element to the user image and run from there , or 2) run tempest from nodepool node. | 16:56 |
| derekh | andreaf2: 1) reuses the tempest element we have and 2) could reuse some devstack stuff (I'm not familliar with the devstack stuff) | 16:57 |
| SpamapS | derekh: I like 2. | 16:58 |
| derekh | andreaf2: either way I'm in favor of doing it now but would like to start off with a warn if there are problem instead of a hard fail, at least until we get an idea of how often things will be blocked | 16:58 |
| SpamapS | derekh: the booted vm is nested is it not? | 16:58 |
| derekh | SpamapS: yes, it would be | 16:58 |
| andreaf2 | derekh: I'd say there could be shared technology between devstack and tripleo (as they both deploy a cloud) but they should not depend one on each other | 16:58 |
| derekh | SpamapS: we could run it from the undercloud either | 16:58 |
| SpamapS | derekh: we could, or just run it from the overcloud controller too. | 16:58 |
| SpamapS | let's just make that image bigger ;) | 16:59 |
| SpamapS | not | 16:59 |
| *** shakayumi has joined #tripleo | 17:00 | |
| *** shakayumi has quit IRC | 17:01 | |
| andreaf2 | so if we run tempest from a VM running with nested virtualization, we need to provide the right venvs already in place or internet connection? | 17:01 |
| *** vkozhukalov has joined #tripleo | 17:01 | |
| andreaf2 | derekh, SpamapS, bnemec: ^^^ | 17:02 |
| *** kiall_ is now known as Kiall | 17:02 | |
| bnemec | andreaf2: The only venv you would need is for tempest. | 17:03 |
| bnemec | I'm not sure we gain much by using devstack though. | 17:03 |
| bnemec | It configures tempest for the environment it installs, so we're going to end up having to do most of the stuff we're doing in the tempest element anyway. | 17:04 |
| andreaf2 | bnemec: yes indeed - so would it be backed into the image? | 17:04 |
| bnemec | andreaf2: You would need to pull down the tempest that matches the services installed in the overcloud. | 17:05 |
| jp_at_hp | Can anyone here supply a 2nd +2 to my patch? Pretty please? https://review.openstack.org/#/c/80644/ | 17:05 |
| derekh | andreaf2: actually maybe running running from the user image isn't a greate idea (could be slow), overcloud controller is a bad idea because ideally tempest wouldn't be running on the same host its testing (we could miss firewall problems ect..), we can't add tempest to the undercloud as we don't have undercloud in the overcloud ci job | 17:06 |
| derekh | derekh: so if we want to use the tempest element then that leaves us with, adding it to the seed | 17:06 |
| * bnemec notes that derekh has started talking to himself :-) | 17:07 | |
| derekh | andreaf2: bnemec : the seed would then have the tempest venv baked into it | 17:07 |
| derekh | bnemec: thanks, I must be going mad | 17:08 |
| bnemec | :-) | 17:08 |
| *** pbelanyi has quit IRC | 17:09 | |
| andreaf2 | derekh: using seed sounds like a good plan | 17:09 |
| *** dprince has quit IRC | 17:11 | |
| *** gcha has quit IRC | 17:11 | |
| andreaf2 | derekh: this may be a good topic for tomorrow's meeting :) | 17:14 |
| *** ifarkas has joined #tripleo | 17:14 | |
| *** rlandy_ has quit IRC | 17:14 | |
| derekh | andreaf2: Yup, agreed | 17:14 |
| *** e0ne has quit IRC | 17:14 | |
| andreaf2 | derekh: I'll try to join tomorrow evening then | 17:15 |
| andreaf2 | derekh: is it on #openstack-meeting-alt ? | 17:16 |
| derekh | andreaf2: cool, I probably wont be there DST screwed over the time for me but will if I can, should be abck attending next week when clocks change | 17:16 |
| derekh | andreaf2: yup | 17:16 |
| *** nati_ueno has joined #tripleo | 17:17 | |
| *** epim has joined #tripleo | 17:18 | |
| *** flashgordon is now known as jogo | 17:34 | |
| *** jistr has quit IRC | 17:36 | |
| *** fandi has joined #tripleo | 17:39 | |
| *** athomas has quit IRC | 17:45 | |
| openstackgerrit | Clint "SpamapS" Byrum proposed a change to openstack/os-collect-config: Parse deployments if found in cfn metadata https://review.openstack.org/82244 | 17:46 |
| *** derekh has quit IRC | 17:54 | |
| *** CaptTofu has joined #tripleo | 17:57 | |
| openstackgerrit | A change was merged to openstack/diskimage-builder: Adds libmariadb-dev package mapping https://review.openstack.org/79874 | 18:02 |
| openstackgerrit | Ana Krivokapic proposed a change to openstack/tuskar-ui: Fix checkboxes on the nodes page https://review.openstack.org/82568 | 18:03 |
| *** dprince has joined #tripleo | 18:14 | |
| openstackgerrit | A change was merged to openstack/tripleo-image-elements: Add -i options to os-svc-install/os-svc-daemon... https://review.openstack.org/76952 | 18:19 |
| *** lucasagomes is now known as lucas-afk | 18:21 | |
| *** andreaf2 has quit IRC | 18:23 | |
| *** morganfainberg_Z is now known as morganfainberg | 18:24 | |
| *** jpeeler has quit IRC | 18:26 | |
| *** mestery has joined #tripleo | 18:27 | |
| *** mestery has quit IRC | 18:29 | |
| *** e0ne has joined #tripleo | 18:30 | |
| *** Hefeweizen has quit IRC | 18:30 | |
| openstackgerrit | Ana Krivokapic proposed a change to openstack/tuskar-ui: Fix checkboxes on the nodes page https://review.openstack.org/82568 | 18:39 |
| lifeless | morning | 18:41 |
| greghaynes | O/ | 18:41 |
| *** akrivoka has quit IRC | 18:47 | |
| *** ramishra has quit IRC | 18:47 | |
| *** ramishra has joined #tripleo | 18:48 | |
| *** martyntaylor has joined #tripleo | 18:48 | |
| *** martyntaylor has left #tripleo | 18:49 | |
| *** ramishra has quit IRC | 18:52 | |
| lifeless | greghaynes: hey! | 18:53 |
| lifeless | greghaynes: so, extradb in, whats next HA wise ? | 18:53 |
| greghaynes | Hoping to get a multinode just galera and everything else on one node config, there are some cluster config options which need to be templated out and initial master node selection | 18:55 |
| greghaynes | config meaning heat template | 18:55 |
| lifeless | rabbit as well, right ? | 18:55 |
| *** jang1 has joined #tripleo | 18:55 | |
| lifeless | I say that because splitting where rabbit and galera are may be tricky in the template | 18:55 |
| greghaynes | I was going to do just galera to get the galera config, then same with rabbit, then with both | 18:56 |
| greghaynes | ah | 18:56 |
| greghaynes | ok, will do that then | 18:56 |
| lifeless | the rabbit HA stuff has landed | 18:56 |
| lifeless | so shouldn't (modulo bugs) need dev work | 18:56 |
| greghaynes | yep, I dont know if its actually been tested multinode yet... | 18:57 |
| greghaynes | although I think youre right it should all be there | 18:57 |
| lifeless | by jprovazn I am quite sure it has | 18:57 |
| greghaynes | woot | 18:57 |
| *** lucas-afk is now known as lucasagomes | 18:57 | |
| jprovazn | greghaynes, lifeless: yep - multinode rabbitmq worked for me | 18:57 |
| greghaynes | I was hoping to also play with software-config a bit today... | 18:57 |
| greghaynes | jprovazn: what did the heat template look like? | 18:58 |
| greghaynes | since I might be able to use it for galera | 18:58 |
| jprovazn | greghaynes, very nasty: https://gist.github.com/jprovaznik/9662754 | 18:58 |
| greghaynes | haha, awesome | 18:58 |
| jprovazn | greghaynes, basically just duplicated node0 to node1 | 18:58 |
| jprovazn | greghaynes, creating scaling template has high prio I think | 18:59 |
| jprovazn | well, there already is basic from lifeless | 18:59 |
| greghaynes | You mean higher prio than software-config? | 18:59 |
| greghaynes | or higher prio than what? | 19:00 |
| lifeless | just hi :) | 19:00 |
| greghaynes | jprovazn: Im surprised your stack create doesnt fail with that config due to on-disk state | 19:02 |
| jprovazn | greghaynes, right, just hi :) - I don't know in compare to software config | 19:02 |
| greghaynes | oh, just clarifying what you meant because I didnt either :p | 19:02 |
| lifeless | so SpamapS has software-config well along | 19:02 |
| greghaynes | yes, I think scaling is the other big high prio | 19:02 |
| lifeless | I think working on top of this is a good idea to avoid rework | 19:02 |
| lifeless | SpamapS: ^ do you concur ? | 19:02 |
| SpamapS | agree yes | 19:03 |
| SpamapS | it turns out we can't lose the explicit wait conditions / handle though... have a re-work in progress for that | 19:03 |
| SpamapS | because of bug 1291905 | 19:04 |
| uvirtbot | Launchpad bug 1291905 in tripleo "new resources are not available for signalling during an update -> heat stack update timeout" [High,Triaged] https://launchpad.net/bugs/1291905 | 19:04 |
| greghaynes | Are there changes needed to merge.py for it to work with software-config? | 19:05 |
| SpamapS | lifeless: note that we have to have one waitcondition for everything because of that | 19:05 |
| SpamapS | greghaynes: yes see https://review.openstack.org/#/c/81666/ | 19:06 |
| greghaynes | oo nice | 19:06 |
| lifeless | reviews on https://review.openstack.org/#/c/82244/ are a pre-requisite | 19:08 |
| lifeless | slagle: bnemec: ^ | 19:08 |
| lifeless | SpamapS: btw https://bugs.launchpad.net/ironic/+bug/1295503 would appreciate any insight you have to handling that through heat | 19:09 |
| uvirtbot | Launchpad bug 1295503 in ironic "ironic nova driver blocks nova-compute startup when ironic isn't available" [Medium,Incomplete] | 19:09 |
| *** jprovazn has quit IRC | 19:10 | |
| openstackgerrit | A change was merged to openstack/os-collect-config: Updated from global requirements https://review.openstack.org/81648 | 19:12 |
| *** CaptTofu has quit IRC | 19:14 | |
| *** CaptTofu has joined #tripleo | 19:15 | |
| bnemec | lifeless: Did you mean to ping me about that? I'm happy to take a look, but it doesn't seem like something I'm any better equipped to review than anyone else. :-) | 19:18 |
| *** UtahDave has joined #tripleo | 19:18 | |
| bnemec | Just thought I'd check in case you meant to bring it to someone else's attention. | 19:18 |
| *** CaptTofu has quit IRC | 19:19 | |
| *** e0ne has quit IRC | 19:25 | |
| lifeless | bnemec: I thought you were core; I need to get that mail out :) | 19:30 |
| lifeless | lsmola_: marios_ shadower: https://review.openstack.org/#/c/82244/ | 19:30 |
| bnemec | Ah, cool. :-) | 19:31 |
| *** hashar has joined #tripleo | 19:36 | |
| *** epim has quit IRC | 19:42 | |
| openstackgerrit | Gregory Haynes proposed a change to openstack/tripleo-image-elements: Explicitly install pbr via pip https://review.openstack.org/81101 | 19:44 |
| *** mestery has joined #tripleo | 19:47 | |
| Lotus907efi | so I am capturing my console output from my devtest.sh runs and I noticed that the phrase "Installing setuptools, pip...done" occurred over 60 times in the log | 19:56 |
| Lotus907efi | does the install of setuptools really happen that many times? | 19:57 |
| lifeless | yah | 19:59 |
| Lotus907efi | wow | 19:59 |
| Lotus907efi | what is this??? : Caching xtradb-cluster tarball from http://www.percona.com/redir/downloads/TESTING/Percona-XtraDB-Cluster/5.5.33-23.7.5/release-5.5.33/476/binary/linux/x86_64/Percona-XtraDB-Cluster-5.5.33-23.7.5.476-static-openssl-1.0.1e.Linux.x86_64.tar.gz | 20:01 |
| bnemec | Lotus907efi: That's for MySQL clustering support | 20:05 |
| *** newell has quit IRC | 20:06 | |
| Lotus907efi | why does it go out to www.percona.com rather than some openstack hosted site? | 20:06 |
| bnemec | Hmm, good point. We probably want to mirror that to infra somewhere so we don't break if their site goes down. | 20:07 |
| *** newell has joined #tripleo | 20:08 | |
| *** lucasagomes has left #tripleo | 20:10 | |
| Ng | bnemec: Lotus907efi: not saying it shouldn't be cached on openstack.org (licensing willing), but afair that is a temporary thing to workaround a bug in current packaged versions | 20:11 |
| * Ng points at the "TESTING" part of the URL :) | 20:11 | |
| Lotus907efi | hmmm, ok | 20:12 |
| lifeless | Ng: thats just the percona stability url | 20:12 |
| Lotus907efi | just curious | 20:12 |
| bnemec | Ng: I think the packaged versions were going to come from percona too though, weren't they? | 20:12 |
| lifeless | Ng: so no, we're actually just pulling a binary because that works on all platforms vs repos which are inconsistent | 20:12 |
| Ng | bnemec: pass | 20:12 |
| lifeless | Ng: once percona get repos for all our platforms we should switch to repos | 20:12 |
| lifeless | Ng: and of course some folk will want mariadb etc etc | 20:12 |
| Ng | :) | 20:13 |
| *** hashar has quit IRC | 20:13 | |
| Ng | "Bad md5 hash for package https://pypi.python.org/packages/source/c/cliff/cliff-1.5.2.tar.gz#md5=650d3155b38158b79563bd0f0327f081 (from https://pypi.python.org/simple/cliff/)" grr | 20:13 |
| bnemec | <3 pypi ;-) | 20:13 |
| *** hashar has joined #tripleo | 20:14 | |
| *** jp_at_hp has quit IRC | 20:15 | |
| lifeless | Ng: GheRivero: did you get some progress on build perf @ HP ? | 20:16 |
| *** noslzzp has quit IRC | 20:17 | |
| greghaynes | Theres also a percona bug requiring us to use testing release... woo working around bugs from working around bugs | 20:18 |
| *** noslzzp has joined #tripleo | 20:18 | |
| openstackgerrit | A change was merged to openstack/os-collect-config: Parse deployments if found in cfn metadata https://review.openstack.org/82244 | 20:21 |
| *** notq has joined #tripleo | 20:28 | |
| *** epim has joined #tripleo | 20:29 | |
| *** marun has quit IRC | 20:32 | |
| openstackgerrit | Chris Jones proposed a change to openstack/tripleo-incubator: Store TE_DATAFILE in tripleorc. https://review.openstack.org/82595 | 20:33 |
| *** rlandy has joined #tripleo | 20:34 | |
| *** marun has joined #tripleo | 20:40 | |
| lifeless | Ng: https://review.openstack.org/#/c/80445/ ? | 20:46 |
| Ng | lifeless: Ghe was looking at timings on the lab hw, I was looking at it locally. I'm sort of wondering if we're just having a disagreement about what "reasonable hardware" means - does that mean "my laptop with an SSD" to you? If so, that is probably going to be better at IO than the rusty platters in the lab machines, and an image build at least, is definitely IO bound | 20:46 |
| *** noslzzp has quit IRC | 20:47 | |
| *** e0ne has joined #tripleo | 20:47 | |
| Ng | lifeless: hah, well I'll just go ahead and abandon mine then :) | 20:48 |
| lifeless | Ng: 20m to copy 6G of data is unreadonable on any machine bought in the last 4 years. | 20:48 |
| lifeless | Ng: even a measly 100MBps will get that one in 60 seconds. | 20:48 |
| lifeless | s/one/done/ | 20:48 |
| lifeless | Ng: no, we're *slow*. | 20:48 |
| lifeless | Ng: image builds happen in tmpfs. How are we IO bound? | 20:49 |
| Ng | lifeless: the mirrors aren't in tmpfs | 20:49 |
| *** vkozhukalov has quit IRC | 20:49 | |
| jang1 | ng: if they're not cached then linux needs a new vm subsystem :-( | 20:49 |
| lifeless | Ng: they aren't, but they really should be in ram after a single run :) | 20:50 |
| lifeless | Ng: also Lotus907efi tried the --offline thing and had a massive perf boost. | 20:50 |
| lifeless | Ng: so we have at least one data point validating the theory. | 20:50 |
| Ng | fwiw, I'm not disagreeing that we're slow, I just wanted to query if we're in the same orders of magnitude with "reasonable hardware" :) | 20:51 |
| jang1 | If I can chip in: the granularity of --offline is pretty large. Any indication which thing it's caching that makes all the difference? | 20:51 |
| *** jpeeler has joined #tripleo | 20:53 | |
| *** jpeeler has quit IRC | 20:54 | |
| *** jpeeler has joined #tripleo | 20:54 | |
| *** jpeeler has joined #tripleo | 20:54 | |
| *** jang1 has quit IRC | 20:55 | |
| *** jang1 has joined #tripleo | 20:56 | |
| *** d0ugal has quit IRC | 20:57 | |
| openstackgerrit | A change was merged to openstack/tripleo-incubator: Source devtest_variables in tripleorc https://review.openstack.org/80445 | 20:57 |
| *** d0ugal has joined #tripleo | 20:58 | |
| *** d0ugal has joined #tripleo | 20:58 | |
| Lotus907efi | lifeless: please see John's questions in the HipChat TripleO room | 21:01 |
| lifeless | jang1: pypi.python.org is removed from pip.conf with --offline | 21:01 |
| * greghaynes suddenly remembers hipchat | 21:03 | |
| jang1 | I should look at the pypi element again; I use PYPI_MIRROR_URL - but that would explain the pypi.python.org pings I've seen bouncnig off squid. | 21:04 |
| greghaynes | You need to do both IIRC | 21:04 |
| greghaynes | use the element + that var | 21:04 |
| jang1 | I use the element | 21:04 |
| jang1 | I'm just wondering if it still inserts an extra-index-url in there | 21:04 |
| greghaynes | ah | 21:05 |
| jang1 | if [ -z $DIB_OFFLINE ]; then | 21:06 |
| jang1 | ONLINE="extra-index-url = https://pypi.python.org/simple" | 21:06 |
| jang1 | else | 21:06 |
| jang1 | ONLINE="" | 21:06 |
| jang1 | fi | 21:06 |
| jang1 | cat <<EOF > ~/.pip/pip.conf | 21:06 |
| jang1 | [global] | 21:06 |
| jang1 | index-url = $PYPIURL | 21:06 |
| jang1 | $ONLINE | 21:06 |
| jang1 | log = $HOME/pip.log | 21:06 |
| jang1 | EOF | 21:06 |
| jang1 | excuse the C&P | 21:06 |
| jang1 | hrm, I'll be nuking that in the morning. | 21:06 |
| *** rlandy has quit IRC | 21:06 | |
| *** bogdando has quit IRC | 21:06 | |
| *** bogdando has joined #tripleo | 21:08 | |
| *** lblanchard has quit IRC | 21:12 | |
| lifeless | jang1: nuking it ? Its deliberate ... | 21:13 |
| lifeless | jang1: making it more controllable +1. | 21:13 |
| Lotus907efi | yeah, if it would be possible to use internal pypi mirror ONLY and internal ubuntu repo but still allow fallback to external ubuntu repo that would be cool | 21:15 |
| *** andreaf has joined #tripleo | 21:15 | |
| Lotus907efi | because I keep discovering things that are not in our internal ubuntu repo that were not a problem before I added "--offline" to the DIB_COMMON_ELEMENTS env | 21:16 |
| jang1 | That's what I meant | 21:17 |
| Lotus907efi | and it keeps stopping the build after it gets almost all of the way through the overcloud image build | 21:17 |
| Lotus907efi | undercloud and overcloud-control builds fine from our internal ubuntu repo | 21:17 |
| openstackgerrit | Saurabh Surana proposed a change to openstack/tripleo-image-elements: base element for trove control plane elements https://review.openstack.org/82605 | 21:18 |
| Lotus907efi | but something in the overcloud compute nodes stops the whole thing because it cannot find something in our repo right at the end | 21:18 |
| jang1 | lifeless: I meant, my intention was to ensure that extra-index-url didn't leak into my config. | 21:18 |
| Lotus907efi | jang1: where did you see that: if [ -z $DIB_OFFLINE ]; then ONLINE="extra-index-url = https://pypi.python.org/simple" and etc? | 21:19 |
| jang1 | Lotus907efi: the problem may well be this..? https://review.openstack.org/#/c/82134/ | 21:19 |
| jang1 | Lotus907efi: that's in the pypi element. | 21:19 |
| jang1 | the link above may be to your problem, depending on how your pypi mirror's built. | 21:20 |
| Lotus907efi | hmm, no I am getting failures when trying to pull stuff from ubuntu repo | 21:20 |
| openstackgerrit | Saurabh Surana proposed a change to openstack/tripleo-image-elements: element for installing Trove-API service https://review.openstack.org/82606 | 21:20 |
| Lotus907efi | not the pypi mirror | 21:20 |
| Lotus907efi | for the overcloud compute image | 21:21 |
| *** e0ne has quit IRC | 21:23 | |
| openstackgerrit | Saurabh Surana proposed a change to openstack/tripleo-image-elements: element for installing Trove-TaskManager service https://review.openstack.org/82610 | 21:23 |
| openstackgerrit | Dan Prince proposed a change to openstack/tripleo-image-elements: Horizon: dynamically set config time env vars https://review.openstack.org/82611 | 21:24 |
| openstackgerrit | Dan Prince proposed a change to openstack/tripleo-image-elements: TEST ONLY: make nova depend on common-venv https://review.openstack.org/79989 | 21:24 |
| openstackgerrit | jan grant proposed a change to openstack/tripleo-image-elements: Ensure the nova element is installable https://review.openstack.org/82134 | 21:25 |
| *** dprince has quit IRC | 21:26 | |
| openstackgerrit | Saurabh Surana proposed a change to openstack/tripleo-image-elements: element for installing Trove-Conductor service https://review.openstack.org/82614 | 21:28 |
| *** e0ne has joined #tripleo | 21:30 | |
| *** hewbrocca has quit IRC | 21:32 | |
| *** akrivoka has joined #tripleo | 21:34 | |
| *** julim has quit IRC | 21:35 | |
| SpamapS | lifeless: hey, so I think it is time to resurrect tripleo-cd | 21:41 |
| SpamapS | lifeless: I have a plan for it hitting UPDATE_FAILED .. which is to do an abandon/adopt cycle on it... so at least we don't have to stack-delete it. | 21:42 |
| *** jpeeler has left #tripleo | 21:44 | |
| *** nati_ueno has quit IRC | 21:47 | |
| *** nati_ueno has joined #tripleo | 21:48 | |
| *** d0ugal has quit IRC | 21:49 | |
| openstackgerrit | A change was merged to openstack/tripleo-heat-templates: Pass undercloud metadata to overcloud compute https://review.openstack.org/81552 | 21:52 |
| *** jtomasek has quit IRC | 21:54 | |
| *** newell has quit IRC | 21:55 | |
| *** mestery has quit IRC | 21:55 | |
| openstackgerrit | Richard Su proposed a change to openstack/tripleo-image-elements: Disable swift container sync service https://review.openstack.org/82625 | 22:00 |
| tchaypo | SpamapS: i hear tell you discovered a race condition involving ovs | 22:03 |
| tchaypo | SpamapS: specifically, lifeless left a comment on https://bugs.launchpad.net/bugs/1290486 | 22:04 |
| uvirtbot | Launchpad bug 1290486 in tripleo "dhcp agent not serving responses" [Critical,In progress] | 22:04 |
| *** mestery has joined #tripleo | 22:04 | |
| *** mestery has quit IRC | 22:06 | |
| SpamapS | tchaypo: yeah, seems neutron doesn't respond well to ovs not being ready yet. | 22:10 |
| *** d0ugal has joined #tripleo | 22:10 | |
| *** noslzzp has joined #tripleo | 22:12 | |
| *** matty_dubs is now known as matty_dubs|gone | 22:13 | |
| tchaypo | SpamapS: the description lifeless put in the ticket makes it sound as though it's a problem getting set up | 22:14 |
| *** jang1 has quit IRC | 22:14 | |
| tchaypo | what I'm seeing is that the system works and then a day later packets aren't being forwarded onto the right interface | 22:14 |
| *** d0ugal has quit IRC | 22:14 | |
| lifeless | tchaypo: oh, thats new :) | 22:14 |
| tchaypo | ah, so it's not just me not understanding how those two things fit together? | 22:15 |
| *** akrivoka has quit IRC | 22:17 | |
| *** eghobo has quit IRC | 22:22 | |
| tchaypo | and I've just confirmed that I can reproduce it on my new server, and once again restart neutron-openvswitch-agent fixes it | 22:22 |
| *** noslzzp has quit IRC | 22:22 | |
| *** ifarkas has quit IRC | 22:31 | |
| *** andreaf has quit IRC | 22:33 | |
| *** howleyt_ has joined #tripleo | 22:35 | |
| *** saurabhs has joined #tripleo | 22:35 | |
| *** e0ne has quit IRC | 22:41 | |
| lifeless | tchaypo: what triggers it? | 22:43 |
| lifeless | tchaypo: also check neutron-openvswitch-agent logs for hints | 22:43 |
| *** CaptTofu has joined #tripleo | 22:48 | |
| tchaypo | looked there, and in /var/log/upstart/neutron*, but the hint that cracked it came from syslog | 22:50 |
| lifeless | what was the hint ? | 22:50 |
| tchaypo | Mar 24 13:25:06 undercloud-undercloud-3tppvbvqvfoc kernel: [24541.522437] ovsdb-server[2983]: segfault at 904 ip b7313231 sp bf80b930 error 4 in libgcc_s.so.1[b72ff000+1b000] | 22:51 |
| tchaypo | Mar 24 13:25:07 undercloud-undercloud-3tppvbvqvfoc ovsdb-server: 00002|daemon(monitor)|ERR|1 crashes: pid 2983 died, killed (Segmentation fault), core dumped, restarting | 22:51 |
| tchaypo | I can recreate it now - if I kill and restart ovsdb-server I get the same symptoms. need to restart neutron-openvswitch-agent | 22:53 |
| SpamapS | segfaultsrrrbaddmmkay | 22:53 |
| tchaypo | let me see if a polite restart causes the same issue or if it's just a problem when it dies hard | 22:54 |
| tchaypo | yep, a very polite " service openvswitch-switch restart" gives the same symptoms | 22:57 |
| *** CaptTofu has quit IRC | 22:58 | |
| tchaypo | 2014-03-24 22:49:27.179 8455 ERROR neutron.agent.linux.ovsdb_monitor [req-d2c2cbd5-a77a-4455-84ac-0a8ec69b41e8 None] Error received from ovsdb monitor: 2014-03-24T22:49:27Z|00001|fatal_signal|WARN|terminating with signal 15 (Terminated) | 22:59 |
| tchaypo | 2014-03-24 22:55:45.441 16033 ERROR neutron.agent.linux.ovsdb_monitor [req-5fe682ce-138e-46d6-aa7e-f0d43ab576ee None] Error received from ovsdb monitor: ovsdb-client: unix:/var/run/openvswitch/db.sock: receive failed (End of file) | 22:59 |
| tchaypo | The first one was when I killed ovsdb-server, the second one was the polite restart | 22:59 |
| *** CaptTofu has joined #tripleo | 22:59 | |
| lifeless | tchaypo: so restartning openvswitch-switch after neutron agent is running breaks things? | 22:59 |
| lifeless | tchaypo: that seems like a fairly clear bug | 23:00 |
| tchaypo | yeah. and it seems like a fairly clear *neutron* bug. | 23:00 |
| StevenK | Hmm, neither the Weston or Omni love me. | 23:01 |
| StevenK | Which saddens me. | 23:01 |
| clarkb | StevenK: ya I am staying in hilton doubletree as a result | 23:01 |
| StevenK | clarkb: Is that close by? | 23:01 |
| clarkb | it was closer than the westin iirc | 23:02 |
| clarkb | but not as close as Omni | 23:02 |
| tchaypo | either I've missed something by not looking at corp email today, or you're planning the trip in advance of confirmation that we're going | 23:02 |
| clarkb | tchaypo: you should read your email | 23:02 |
| * tchaypo reads email | 23:02 | |
| SpamapS | 2014-03-24 20:16:51.224 | Fetched 5593 kB in 5s (935 kB/s) | 23:03 |
| SpamapS | 2014-03-24 20:16:51.262 | W: Failed to fetch bzip2:/var/lib/apt/lists/partial/archive.ubuntu.com_ubuntu_dists_saucy-updates_main_binary-amd64_Packages Hash Sum mismatch | 23:03 |
| SpamapS | hadn't seen that one in a tripleo run | 23:03 |
| * SpamapS recheck no bugs | 23:03 | |
| SpamapS | Can't believe apt still can't do atomic archive updates :-P | 23:03 |
| *** CaptTofu has quit IRC | 23:03 | |
| StevenK | One of the frontends might be syncing | 23:03 |
| tchaypo | very good. will handle that today. | 23:04 |
| StevenK | tchaypo: Heh, I started planning last night | 23:04 |
| StevenK | But that is because I want to catch up with someone in Dallas. | 23:04 |
| lifeless | jogo: you know tripleo-ci isn't a third party system right ? | 23:04 |
| * greghaynes is in doubletree as well | 23:04 | |
| StevenK | greghaynes, clarkb: All done through CWT? | 23:05 |
| greghaynes | yep | 23:05 |
| StevenK | Excellent. | 23:05 |
| StevenK | I shall sort that out after some breakfast. | 23:05 |
| *** CaptTofu has joined #tripleo | 23:07 | |
| jogo | lifeless: I do | 23:07 |
| lifeless | jogo: I had some cognitive dissonance when I read 'like all the other third party...' in your mail | 23:08 |
| jogo | lifeless: yes I didn't make that clear | 23:09 |
| jogo | I was thinking it should need to meet the same reqs for a 3rd party system to be non-voting check only though | 23:09 |
| lifeless | wheee | 23:09 |
| lifeless | cp: '/home/robertc/work/user.qcow2-new': failed to get extents info: No such file or directory | 23:09 |
| lifeless | ewtf | 23:09 |
| jogo | as in for a 3rd party system if it falls over and doesn't work nova doesn't stop | 23:10 |
| lifeless | and no, thats not a disk space problem | 23:10 |
| lifeless | /dev/mapper/lifelesswks--vg-lifelesswks--lvs 15T 7.1T 7.2T 50% / | 23:10 |
| jogo | becasue the jobs are non-voting and all | 23:10 |
| Lotus907efi | can anyone tell me why I would want DIB_APT_SOURCES=/etc/apt/sources.list (where the sources are an internal repo mirror) and have DIB_DISTRIBUTION_MIRROR set to point to it too? | 23:10 |
| lifeless | Lotus907efi: pick one :) | 23:11 |
| Lotus907efi | i.e. do I need both or just one or the other? | 23:11 |
| Lotus907efi | well late in the build of the overcould compute image it keeps falling back to pulling stuff from archive.ubuntu.com | 23:11 |
| Lotus907efi | instead of my mirror | 23:11 |
| Lotus907efi | and I am trying to figure out why | 23:12 |
| tchaypo | lifeless, SpamapS - see my latest comment on https://bugs.launchpad.net/tripleo/+bug/1290486 - in order to get neutron to take a look at this, can I just use the "also affects project" thingy? | 23:12 |
| uvirtbot | Launchpad bug 1290486 in tripleo "dhcp agent not serving responses" [Critical,In progress] | 23:12 |
| lifeless | tchaypo: yes | 23:14 |
| lifeless | Lotus907efi: got a trace to pastebin ? | 23:14 |
| Lotus907efi | yes, I always collect traces | 23:14 |
| greghaynes | Lotus907efi: apt-sources looks like it should override the DIB_DISTRIBUTION_MIRROR | 23:15 |
| Lotus907efi | now I have to figure out how I get pastebin to work again with such a large file | 23:15 |
| greghaynes | so should have no effect if you set both... | 23:15 |
| lifeless | Ng: haha https://review.openstack.org/#/c/63021/5/elements/source-repositories/README.md merged with whitespace | 23:16 |
| lifeless | clarkb: how do you map from a change id to e.g. refs/changes/72/61972/8 ? | 23:18 |
| clarkb | lifeless: search for change id in gerrit | 23:18 |
| clarkb | if using web UI just drop the change id in the search box | 23:19 |
| clarkb | you can also do it with the ssh query api. something like ssh review.o.o gerrit query $CHANGEID | 23:19 |
| *** hashar has quit IRC | 23:19 | |
| lifeless | clarkb: I know how to get to the web page | 23:19 |
| lifeless | is the refs/changes/xx/abcde/y value specifically i want | 23:20 |
| lifeless | has anyone else found they can't ctrl-f usefully on gerrit pages nowadays ? | 23:20 |
| clarkb | lifeless: right you can get it on the webpage or via the ssh api | 23:21 |
| greghaynes | easy_install! I stab at thee | 23:22 |
| greghaynes | similar issue to pbr for cffi if its installed as a dependency :( | 23:22 |
| * SpamapS -> kids | 23:22 | |
| openstackgerrit | A change was merged to openstack/tripleo-image-elements: Ensure the nova element is installable https://review.openstack.org/82134 | 23:23 |
| openstackgerrit | A change was merged to openstack/tripleo-image-elements: Explicitly install pbr via pip https://review.openstack.org/81101 | 23:24 |
| *** cwolferh has quit IRC | 23:25 | |
| Lotus907efi | lifeless: http://paste.openstack.org/show/74202/ | 23:25 |
| *** CaptTofu has quit IRC | 23:25 | |
| Lotus907efi | trace of late in overcloud compute node image build systems starts pulling from external ubuntu archive | 23:25 |
| *** CaptTofu has joined #tripleo | 23:25 | |
| Lotus907efi | instead of my internal mirror | 23:26 |
| Lotus907efi | prior to that it is successfully pulling from internal ubuntu mirror | 23:26 |
| Lotus907efi | greghaynes: are you talking about a problem with proxies and easy_install again? | 23:28 |
| greghaynes | Yes, if your pypi-mirror is recent the fix for mirroring normal package even if a wheel is made should let your avoid this issue (I think) | 23:29 |
| *** cwolferh has joined #tripleo | 23:29 | |
| greghaynes | but this does prevent me from using wheel based pip mirror for great speed | 23:30 |
| *** CaptTofu has quit IRC | 23:30 | |
| lifeless | Lotus907efi: | 23:30 |
| lifeless | + /root/.cache/tripleo/diskimage-builder/bin/disk-image-create ubuntu vm -a amd64 -o /root/.cache/tripleo/user | 23:30 |
| lifeless | Lotus907efi: we don't apply any of the COMMON stuff to the user test image. | 23:30 |
| lifeless | Lotus907efi: we shouldn't really either, because its meant to be a basic reference image. arguably we should switch that to cirros. | 23:31 |
| *** saurabhs has quit IRC | 23:31 | |
| clarkb | greghaynes: not if you set things in ~/.pip/pip.conf properly | 23:31 |
| lifeless | Lotus907efi: anyhow, look near the end of devtest_overcloud.sh, should be easy to spot. | 23:31 |
| Lotus907efi | hmm, I must be dense today. I am not understanding this even after reading your comments several times now :( | 23:32 |
| Lotus907efi | ok | 23:32 |
| Lotus907efi | I will look at that | 23:32 |
| clarkb | greghaynes: eg http://paste.openstack.org/show/74204/ | 23:32 |
| clarkb | greghaynes: first index-url points at wheels, the other two fall back to normal packages | 23:33 |
| Lotus907efi | lifeless: what does the end user image do? what is it used for? | 23:35 |
| clarkb | greghaynes: fwiw I am using that on saucy and haven't seen any terrible blowups but I amy be playing with fire | 23:36 |
| *** jroll has joined #tripleo | 23:36 | |
| jroll | should check-tripleo-ironic-undercloud-precise currently be passing or failing? | 23:37 |
| *** yamahata has quit IRC | 23:39 | |
| greghaynes | clarkb: hrm, ill mess with that and see if it fixes | 23:41 |
| clarkb | greghaynes: if you build your own mirror it should have the proper suffix for your distro | 23:42 |
| Lotus907efi | so if I set DIB_DISTRIBUTION_MIRROR to point to my internal ubuntu repo will the end user test image also be built from that mirror or will it still try to go to the outside world? | 23:43 |
| greghaynes | clarkb: Yea, I just need to make element aware that im trying to do a wheel based mirror and write a sane config file | 23:45 |
| lifeless | Lotus907efi: its used to boot a vm in the qemu (kvm for real hardware) cloud to show it works | 23:47 |
| Lotus907efi | so does it have to be built from archive.ubuntu.blah or can I get the system to use my internal repo? | 23:48 |
| lifeless | you can build it from your repo | 23:48 |
| lifeless | you'll need to patch devtest_overcloud though | 23:48 |
| lifeless | since its not currently exposed as a configurable thing | 23:48 |
| Lotus907efi | hmmm | 23:49 |
| lifeless | you could try exporting NODE_DIST="ubuntu apt-sources" | 23:53 |
| lifeless | looking at | 23:53 |
| lifeless | $TRIPLEO_ROOT/diskimage-builder/bin/disk-image-create $NODE_DIST vm \ | 23:53 |
| Lotus907efi | yeah | 23:54 |
| Lotus907efi | what was the environment variable you were telling I had to set to try using precise? | 23:58 |
| StevenK | Lotus907efi: DIB_RELEASE | 23:59 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!