| openstackgerrit | Gregory Haynes proposed a change to openstack/tripleo-image-elements: Explicitly pip install pbr before its installed as dependency https://review.openstack.org/81101 | 00:01 |
|---|---|---|
| Lotus907efi | nope, do this test: completely remove you tripleo-incubator sources and the ~/.cache/tripleo directory | 00:01 |
| Lotus907efi | and then pull the tripleo-incubator code as if you were starting from scratch on a new system | 00:02 |
| Lotus907efi | where does the tripleo-image-elements code get pulled by the devtest.sh script execution? | 00:03 |
| greghaynes | in _setup and _refresh_env | 00:03 |
| greghaynes | but you should manually have checked out that review inwhich case it wont pull that repo | 00:04 |
| Lotus907efi | ok, but the install of python-cinderclient seems to take place before that | 00:04 |
| Lotus907efi | because I do not have a tripleo-image-elements directory under ~/.cache/tripleo/ yet | 00:04 |
| greghaynes | aye - I think this is a place where my bug doesnt match yours :( | 00:04 |
| Lotus907efi | and the python-cinderclient stuff is running and failing | 00:05 |
| greghaynes | can you paste me the log around where it happens? | 00:05 |
| Lotus907efi | sure | 00:05 |
| greghaynes | on some kind of paste site :) | 00:05 |
| greghaynes | actually - I think your fail might be a different patch | 00:06 |
| lifeless | tchaypo: this is why https://etherpad.openstack.org/p/tripleo-incubator-rationalise-ui | 00:07 |
| greghaynes | for tripleo-incubator | 00:07 |
| Lotus907efi | I just attached the bzip | 00:07 |
| greghaynes | ok | 00:08 |
| Lotus907efi | ed file to the bug | 00:08 |
| Lotus907efi | https://bugs.launchpad.net/tripleo/+bug/1293812 | 00:08 |
| openstackgerrit | Gregory Haynes proposed a change to openstack/tripleo-incubator: Explicitly pip install pbr before its installed as dependency https://review.openstack.org/81126 | 00:09 |
| greghaynes | Lotus907efi: try that for your tripleo-incubator repo as well | 00:10 |
| Lotus907efi | ok | 00:10 |
| Lotus907efi | nope, still failed | 00:12 |
| Lotus907efi | I checked to make sure the change was there: root@ops-ae1az1-tripooo:/opt/git-repo/tripleo-incubator# cat scripts/setup-clienttools |grep pbr | 00:13 |
| Lotus907efi | pip install -U 'pbr>=0.5.21,<1.0' | 00:13 |
| greghaynes | hrm, just looked at your log and it happened at a different spot | 00:13 |
| Lotus907efi | want a new log? | 00:13 |
| lifeless | Lotus907efi: did you preserve your .pip/pip.conf + .pydistutils.cfg files when you reset stuff ? | 00:13 |
| lifeless | Lotus907efi: + your http proxy variables etc? | 00:13 |
| Lotus907efi | yes on the .pydistutils.cfg but I don't think I ever had a .pip/pip.conf file | 00:14 |
| Lotus907efi | let me check | 00:14 |
| Lotus907efi | nope, no ~/.pip/pip.conf file | 00:14 |
| Lotus907efi | but the ~/.pydistutils.cfg file is still there | 00:15 |
| Lotus907efi | and the proxy variables are there too | 00:15 |
| lifeless | so for pip to work in your local environment you need ~/.pip/pip.conf continaining | 00:15 |
| lifeless | [global] | 00:15 |
| lifeless | index-url = http://15...... | 00:15 |
| Lotus907efi | hmmm | 00:15 |
| Lotus907efi | ok | 00:15 |
| Lotus907efi | so same contents as ~/.pydistutils.cfg except word global in square brackets instead of [easy_install] | 00:16 |
| lifeless | and - instead of _ | 00:16 |
| lifeless | read diskimage-builder/elements/pypi/pre-install.d/00-configure-pypi-mirror for the gory details | 00:16 |
| *** julim has joined #tripleo | 00:17 | |
| Lotus907efi | oh, nuts | 00:18 |
| openstackgerrit | James Slagle proposed a change to openstack/tripleo-image-elements: Add fedora-rdo-icehouse element https://review.openstack.org/80658 | 00:19 |
| lifeless | slagle: so did you do a patch as well as arosen for neutron - I saw his push up - should I review that? | 00:20 |
| Lotus907efi | hmm, my ~/.pip/pip.conf file now has: | 00:20 |
| Lotus907efi | [global] | 00:20 |
| Lotus907efi | index-url = http://15.126.28.132/pypi/mirror/ | 00:20 |
| Lotus907efi | but setuptools is now failing to install | 00:20 |
| arosen | if slagle has a patch he probably fixed the issue. There was a part that I got stuck on that i need to come back to. | 00:20 |
| arosen | lifeless: | 00:20 |
| Lotus907efi | Could not find any downloads that satisfy the requirement setuptools>=1.0 | 00:21 |
| Lotus907efi | Cleaning up... | 00:21 |
| Lotus907efi | No distributions at all found for setuptools>=1.0 | 00:21 |
| lifeless | arosen: cool. I'm kindof-not-here today, so just dipping into the flow as time permits | 00:21 |
| greghaynes | Lotus907efi: sounds like invalid mirror | 00:21 |
| lifeless | Lotus907efi: check it is in your mirror ? | 00:21 |
| Lotus907efi | yeah, I am looking now | 00:22 |
| *** julim has quit IRC | 00:24 | |
| Lotus907efi | it is there. I think it is another damn proxy problem | 00:25 |
| lifeless | arosen: the nova admin we use is 'admin' | 00:26 |
| lifeless | arosen: we don't have - and can't have - the tenant uuid until after the deploy. | 00:26 |
| lifeless | arosen: the tenants is service | 00:26 |
| Lotus907efi | does pip use no_proxy=blah,blah ??? | 00:26 |
| *** matsuhashi has joined #tripleo | 00:27 | |
| lifeless | Lotus907efi: it uses requests I believe, which honors no_proxy | 00:27 |
| lifeless | Lotus907efi: whether it does so correctly is a different question | 00:27 |
| Lotus907efi | my no_proxy env vari looks like: no_proxy=192.0.2.0/24,15.126.28.128/27 | 00:27 |
| Lotus907efi | maybe it is not picking up the stuff after the comma correctly? | 00:28 |
| Lotus907efi | let me try explicitly specifying just the mirror host IP | 00:29 |
| greghaynes | Ive never used a subnet for that var before... | 00:30 |
| greghaynes | my initial googling suggests thats not correct | 00:30 |
| Lotus907efi | nuts, that worked | 00:30 |
| Lotus907efi | hmmm | 00:30 |
| lifeless | arosen: is nova_admin_auth_url the keystone URL or nova URL ? | 00:31 |
| greghaynes | yes, so if you really need a full list you can probably get clever with printf | 00:31 |
| lifeless | well | 00:32 |
| lifeless | CIDR is supported by some tools I believe | 00:32 |
| lifeless | but yeah | 00:32 |
| Lotus907efi | hmm, no pip tar.gz file under pypi/mirror/pip/ on my mirror system | 00:33 |
| Lotus907efi | so the mirror must be incomplete somehow | 00:34 |
| *** xuhaiwei has quit IRC | 00:34 | |
| lifeless | arosen: reviewed, hopefully usefully :) | 00:34 |
| Lotus907efi | all that is there is a pip-1.5.4-py2.py3-none-any.whl file and an index.html | 00:35 |
| lifeless | Lotus907efi: you said the mirror was failing at mirroring :) | 00:35 |
| lifeless | Lotus907efi: you're looking for setuptools, not pip, right ? | 00:35 |
| Lotus907efi | yeah, but that was on httpretty | 00:35 |
| lifeless | ok, AFK for a bit | 00:35 |
| Lotus907efi | setuptools worked | 00:35 |
| Lotus907efi | once I correctly specified the no_proxy thing to just the mirror IP | 00:36 |
| Lotus907efi | instead of a network | 00:36 |
| *** CaptTofu has joined #tripleo | 00:37 | |
| *** xuhaiwei has joined #tripleo | 00:38 | |
| Lotus907efi | now now in various pythin-blah client tools whenever pbr gets pulled in then the install of pbr fails because my mirror does not seem to have a copy of pip tar.gz file in pypi/mirror/pip/ | 00:40 |
| Lotus907efi | nuts | 00:40 |
| Lotus907efi | various python-blah client tools | 00:41 |
| Lotus907efi | python-novaclient, python-heatclient, python-glanceclient, and etc | 00:41 |
| *** tserong_ is now known as tserong | 00:42 | |
| greghaynes | Yep, sounds like time to hit youre mirroring tool with a hammer? | 00:43 |
| tchaypo | If only we were in hipchat | 00:45 |
| tchaypo | greghaynes could do a quick s/youre/your/ and the rest of us wouldn't have to wince | 00:46 |
| greghaynes | Thanks for youre help :p | 00:46 |
| StevenK | Argh! | 00:46 |
| StevenK | greghaynes: your a terrible person | 00:47 |
| * StevenK twitches at continuing the badness | 00:47 | |
| * greghaynes -> dinner | 00:49 | |
| Lotus907efi | hmm, perhaps I was wrong the actual pip error seems to be happening as a result of installing pystache | 00:49 |
| Lotus907efi | but it is still saying that there is no suitable install candidate for pip >1,0 | 00:53 |
| *** victor_lowther has quit IRC | 00:53 | |
| *** tteggel has quit IRC | 00:54 | |
| *** victor_lowther has joined #tripleo | 00:55 | |
| *** tteggel has joined #tripleo | 00:56 | |
| *** CaptTofu has quit IRC | 00:59 | |
| *** CaptTofu has joined #tripleo | 00:59 | |
| *** blamar has joined #tripleo | 01:00 | |
| *** CaptTofu has quit IRC | 01:03 | |
| *** CaptTofu has joined #tripleo | 01:07 | |
| lifeless | Lotus907efi: pastebin please | 01:10 |
| *** victor_lowther has quit IRC | 01:12 | |
| Lotus907efi | well I went and manually downloaded pip-1.5.4.tar.gz from the pypi tree and then manually added it with a md5 sum to the index.html in pypi/mirror/pip/ | 01:12 |
| Lotus907efi | and now devtest.sh is getting past that point | 01:12 |
| *** tteggel has quit IRC | 01:13 | |
| Lotus907efi | but this still means the mirror is not complete | 01:13 |
| Lotus907efi | and if I do a "run-mirror" again will it erase my changes? | 01:13 |
| Lotus907efi | (which BTW I should not have had to do manually) | 01:16 |
| *** tteggel has joined #tripleo | 01:16 | |
| *** victor_lowther has joined #tripleo | 01:16 | |
| *** CaptTofu has quit IRC | 01:18 | |
| *** rpodolyaka1 has joined #tripleo | 01:25 | |
| *** CaptTofu has joined #tripleo | 01:27 | |
| *** morganfainberg is now known as morganfainberg_Z | 01:30 | |
| *** nosnos has joined #tripleo | 01:31 | |
| *** rpodolyaka1 has quit IRC | 01:32 | |
| lifeless | Lotus907efi: run-mirror is only additive | 01:33 |
| lifeless | (vs e.g. bandersnatch which is a replica based system) | 01:34 |
| Lotus907efi | ok | 01:34 |
| lifeless | Lotus907efi: please file a bug about run-mirror | 01:34 |
| lifeless | Lotus907efi: are you running it by hand install, or via the tripleo element for maintaining a mirror? If the former, file it on openstack-ci, if the latter, on tripleo. | 01:34 |
| Lotus907efi | ok, I have one already from earlier today regarding the failed build of the httpretty module | 01:34 |
| Lotus907efi | but I can do another one for this | 01:34 |
| lifeless | please do | 01:34 |
| Lotus907efi | I am doing the run-mirror command by hand | 01:35 |
| lifeless | ok,so file it on openstack-ci | 01:35 |
| Lotus907efi | yeah, that is what I did the earlier one today against | 01:35 |
| *** nosnos has quit IRC | 01:38 | |
| *** nosnos has joined #tripleo | 01:38 | |
| *** morganfainberg_Z is now known as morganfainberg | 01:49 | |
| killer_prince | does dib have support for building images for armhf arch..? it seems it fails.. | 01:50 |
| greghaynes | wah | 01:51 |
| greghaynes | Never heard of anyone trying that, but sounds awesome | 01:51 |
| lifeless | killer_prince: it worked for me with qemu-arm installed | 01:53 |
| lifeless | killer_prince: but its not exercised in CI so it could well have bitrotten | 01:54 |
| killer_prince | okay.. let me try.. i am assuming that you used ubuntu 13.10 as base image.. | 01:54 |
| lifeless | no idea ;) | 01:56 |
| killer_prince | lifeless: i tried installing qemu-arm on ubuntu but its unable to locate package.. E: Unable to locate package qemu-arm | 01:57 |
| lifeless | qemu-system-arm perhaps? | 01:57 |
| killer_prince | yup.. just found out.. | 01:58 |
| lifeless | greghaynes: you're on top of the issues Lotus907efi is having right ? | 01:59 |
| greghaynes | Yep, its a bit late but I should be around for a few hours more at least | 02:00 |
| lifeless | greghaynes: if I may make two minor suggestions; a) ping me when you've got the pip install -U pbr patch revamped; b) if you setup a local mirror listening on 192.168.122.1 , remove your default route and use a squid proxy for internet access, you should be pretty close to Lotus907efi's environment | 02:01 |
| greghaynes | lifeless: I pushed a patch where I grepped for pip install and tried to hit everything that seemed resonable | 02:01 |
| * greghaynes plays with mirror setup | 02:02 | |
| lifeless | greghaynes: it failed on undercloud though | 02:03 |
| greghaynes | ugh | 02:03 |
| lifeless | greghaynes: which is not the systemic failure that arosen's patch is for | 02:03 |
| greghaynes | Yes, it looks like it did fail undercloud | 02:04 |
| lifeless | greghaynes: reviewed | 02:06 |
| lifeless | ah | 02:07 |
| lifeless | url: (7) couldn't connect to host | 02:07 |
| lifeless | network glitch | 02:07 |
| lifeless | so right, patch is probably ok; but I do have a request in the review for you. | 02:07 |
| greghaynes | Yerp, swapping comment now | 02:07 |
| *** morganfainberg is now known as morganfainberg_Z | 02:08 | |
| *** killer_prince has quit IRC | 02:10 | |
| *** CaptTofu has quit IRC | 02:11 | |
| greghaynes | haha, almost pasted in your response directly with the see bug 12345 intact | 02:11 |
| lifeless | greghaynes: LOL | 02:12 |
| *** blamar has quit IRC | 02:13 | |
| *** xuhaiwei has quit IRC | 02:14 | |
| openstackgerrit | Gregory Haynes proposed a change to openstack/tripleo-image-elements: Explicitly pip install pbr before its installed as dependency https://review.openstack.org/81101 | 02:19 |
| greghaynes | cmon, no network glitch this time | 02:20 |
| openstackgerrit | Gregory Haynes proposed a change to openstack/tripleo-image-elements: Explicitly pip install pbr before its installed as dependency https://review.openstack.org/81101 | 02:22 |
| *** rpodolyaka1 has joined #tripleo | 02:28 | |
| *** rpodolyaka1 has quit IRC | 02:33 | |
| *** hewbrocc` has joined #tripleo | 02:39 | |
| *** hewbrocca has quit IRC | 02:39 | |
| *** noslzzp has joined #tripleo | 03:00 | |
| *** CaptTofu has joined #tripleo | 03:03 | |
| lifeless | cody-somerville: btw the screen on this laptop is great; I can now do IRC + 2 80-column text windows at once, horizonally - a huge productivity boost | 03:11 |
| *** matsuhashi has quit IRC | 03:14 | |
| *** killer_prince has joined #tripleo | 03:19 | |
| *** matsuhashi has joined #tripleo | 03:20 | |
| *** hewbrocc` has quit IRC | 03:23 | |
| openstackgerrit | Steve Kowalik proposed a change to openstack/tripleo-incubator: Store seed details in the JSON https://review.openstack.org/79051 | 03:25 |
| *** sdake_ has joined #tripleo | 03:26 | |
| *** matsuhashi has quit IRC | 03:30 | |
| SpamapS | lifeless: waaaannntt | 03:32 |
| SpamapS | I pretty much never take my laptop out of my office with this laptop.. | 03:32 |
| SpamapS | because the 24" IPS display I ordered off SmartBuy is just light years better | 03:33 |
| *** matsuhashi has joined #tripleo | 03:33 | |
| StevenK | SpamapS: I tend to use my desktop, which has a 24" IPS ASUS | 03:34 |
| * SpamapS goes green thinking about having a.. blech.. desktop ;) | 03:34 | |
| StevenK | SpamapS: I like gaming | 03:34 |
| StevenK | And gaming on a laptop is just blech :-) | 03:35 |
| lifeless | SpamapS: 1080p :) | 03:35 |
| StevenK | lifeless: Has arosen EOD'd? | 03:36 |
| SpamapS | StevenK: play older games | 03:37 |
| SpamapS | TF2 is all you really need. :) | 03:37 |
| lifeless | StevenK: I have to assume so | 03:38 |
| lifeless | StevenK: if you wanted to fixup his patch .... | 03:38 |
| StevenK | SpamapS: Hm, I should probably add you on Steam | 03:38 |
| StevenK | lifeless: Hence my asking | 03:38 |
| StevenK | I want to land stuff :-) | 03:38 |
| SpamapS | are we not able to demand a quick revert while the right way wends its way through review/CI? | 03:39 |
| StevenK | It's a nova patch that broke us, and they're pretty frozen, aren't they? | 03:40 |
| SpamapS | frozen doesn't mean "we won't fix horrible bugs" | 03:41 |
| StevenK | Sure, but there's a risk when reverting too | 03:41 |
| lifeless | SpamapS: the patch in question fixes a major race condtion that broke the gate for weeks | 03:42 |
| lifeless | SpamapS: and there's a trivial config setting to disable it | 03:42 |
| *** matsuhashi has quit IRC | 03:42 | |
| lifeless | SpamapS: so reverting in nova is just a pointless circumlocation | 03:42 |
| SpamapS | lifeless: ah, so we get to decide between 100% fail rate of our automation and non-deterministic races for users? | 03:42 |
| SpamapS | lifeless: or do we have a workaround too? | 03:42 |
| SpamapS | I've been trying to stay out of this as I do actually want to, at some point, finish my dive into graceful shutdown for heat.. which I think may actually require us to switch to using SoftwareConfig resources instead of LaunchConfigurations btw.. :-P | 03:43 |
| lifeless | SpamapS: we have a candidate patch from arosen | 03:45 |
| lifeless | SpamapS: see tie | 03:45 |
| StevenK | lifeless: So I've got the patch locally, but I'm not sure how to get the id of a tenant in the conf file | 03:45 |
| StevenK | Haha | 03:46 |
| lifeless | StevenK: we can't use the tenant id without having multiple heat stack operations | 03:46 |
| StevenK | The header for that file is laughably out of date | 03:46 |
| StevenK | ## Please submit patches for any setting that should be deleted or Heat-configurable. | 03:46 |
| StevenK | ## https://github.com/openstack/diskimage-builder | 03:46 |
| lifeless | StevenK: does it work if you just use 'service' rather than a tenant id per my comment in the review ? | 03:46 |
| StevenK | lifeless: How do I test it? | 03:47 |
| *** blamar has joined #tripleo | 03:47 | |
| lifeless | StevenK: devtest.sh --trash-my-machine -c ; let it get up and fail to deploy the first vm; then nova delete demo; reconfigure *neutron* on the overcloud network node; then try to boot a new instance in the overcloud | 03:48 |
| lifeless | SpamapS: workaround for us would be to disable waiting for events - see what dansmith said in #openstack-dev about this. | 03:48 |
| lifeless | SpamapS: so yes, we can have a workaround. +1 on you staying out of it. No heroes. OTOH :(. But \o/ stevenk | 03:49 |
| StevenK | lifeless: Can I destroy all mentions of github in tie? | 03:49 |
| lifeless | StevenK: I guess. | 03:50 |
| SpamapS | lifeless: trouble is I'm wasting time trying to get a devtest up to test my software-config converted templates. :-P | 03:50 |
| lifeless | StevenK: sight unseen, I don' tknow :) | 03:50 |
| StevenK | lifeless: See the comment I pasted above | 03:50 |
| lifeless | SpamapS: apply the trivial disable workaround | 03:50 |
| lifeless | StevenK: comments like that sure, easy fix | 03:50 |
| SpamapS | linked in bug 1293782 ? | 03:50 |
| lifeless | SpamapS: #openstack-dev, when I pinged you, after that dansmith chimed up. Read it. | 03:51 |
| StevenK | lifeless: Destroy them totally, or fix to git.o.o ? | 03:51 |
| StevenK | And the correct project, of course | 03:51 |
| lifeless | StevenK: I'd start with fix :) | 03:51 |
| *** morganfainberg_Z is now known as morganfainberg | 03:52 | |
| SpamapS | lifeless: I'm lost, but I have plenty of email and a few expense reports to catch up on so I'll just shift gears back to busy work until I can use devtest again. :-P | 03:54 |
| *** nosnos has quit IRC | 03:58 | |
| *** vkozhukalov_ has joined #tripleo | 04:07 | |
| *** ramishra has joined #tripleo | 04:07 | |
| *** ramishra_ has joined #tripleo | 04:08 | |
| openstackgerrit | Steve Kowalik proposed a change to openstack/tripleo-image-elements: Remove many references to github https://review.openstack.org/81167 | 04:09 |
| *** ramishra has quit IRC | 04:12 | |
| *** akuznetsov has joined #tripleo | 04:16 | |
| *** fandi has joined #tripleo | 04:21 | |
| *** killer_prince has quit IRC | 04:23 | |
| *** rpodolyaka1 has joined #tripleo | 04:30 | |
| *** rpodolyaka1 has quit IRC | 04:34 | |
| StevenK | % nova delete demo | 04:40 |
| StevenK | No server with a name or ID of 'demo' exists. | 04:40 |
| StevenK | lifeless: ^ | 04:40 |
| lifeless | StevenK: (overcloud) ? | 04:41 |
| StevenK | lifeless: Command I ran before that was . ./overcloudrc | 04:41 |
| *** vkozhukalov_ has quit IRC | 04:43 | |
| *** lazy_prince has joined #tripleo | 04:45 | |
| *** CaptTofu has quit IRC | 04:45 | |
| *** CaptTofu has joined #tripleo | 04:46 | |
| StevenK | lifeless: I guess overcloud-notcompute0-rwp6vujxg46l is the overcloud network node you were talking about? | 04:46 |
| lifeless | yes | 04:49 |
| *** CaptTofu has quit IRC | 04:50 | |
| StevenK | lifeless: I edit /etc/neutron/neutron.conf directly, or find the template and then do other stuff? | 04:50 |
| lifeless | StevenK: up to you :) | 04:53 |
| StevenK | lifeless: How do I expand the variables if I edit the conf directly? | 04:53 |
| lifeless | by typing in the values? | 04:54 |
| * StevenK digs it out what looks right from nova.conf | 04:55 | |
| *** matsuhashi has joined #tripleo | 04:55 | |
| StevenK | lifeless: Then restart neutron-server ? | 04:55 |
| lifeless | yes | 04:56 |
| *** nosnos has joined #tripleo | 04:59 | |
| StevenK | Hm, I thought 'nova boot demo' would just work | 04:59 |
| lifeless | StevenK: source overcloudrc-user | 05:03 |
| lifeless | StevenK: and look at the end over overcloud.sh for an example | 05:03 |
| * StevenK looks for the user password first | 05:05 | |
| lifeless | StevenK: os-overcloud-passwords IIRC | 05:05 |
| StevenK | Ah ha | 05:06 |
| StevenK | I see demo | 05:06 |
| *** rcarrillocruz1 has quit IRC | 05:14 | |
| StevenK | lifeless: The new floating IP works. | 05:21 |
| StevenK | ubuntu@demo:~$ uptime | 05:23 |
| StevenK | 05:23:02 up 14 min, 1 user, load average: 0.31, 0.80, 1.06 | 05:23 |
| openstackgerrit | James Polley proposed a change to openstack/tripleo-image-elements: Make os-refresh-config install.d hook run after pip-and-virtualenv https://review.openstack.org/81174 | 05:26 |
| openstackgerrit | Thomas Goirand proposed a change to openstack/tuskar-ui: Use "python -m coverage" rather than use "coverage" explicitly https://review.openstack.org/80762 | 05:26 |
| openstackgerrit | Steve Kowalik proposed a change to openstack/tripleo-image-elements: Add configuration required for neutron-nova events https://review.openstack.org/81120 | 05:27 |
| openstackgerrit | James Polley proposed a change to openstack/tripleo-image-elements: Make os-refresh-config install.d hook run after pip-and-virtualenv https://review.openstack.org/81174 | 05:30 |
| *** rpodolyaka1 has joined #tripleo | 05:30 | |
| openstackgerrit | James Polley proposed a change to openstack/tripleo-image-elements: Make os-refresh-config install.d hook run later https://review.openstack.org/81174 | 05:30 |
| *** rpodolyaka1 has quit IRC | 05:35 | |
| *** akuznetsov has quit IRC | 05:39 | |
| *** ramishra_ has quit IRC | 05:46 | |
| *** ramishra has joined #tripleo | 05:53 | |
| *** rpodolyaka1 has joined #tripleo | 06:01 | |
| *** akuznetsov has joined #tripleo | 06:06 | |
| lifeless | StevenK: so it booted | 06:08 |
| StevenK | lifeless: Yeah | 06:08 |
| lifeless | StevenK: sounds like that works. | 06:08 |
| StevenK | lifeless: Yup. Is your plan to +A the tie change in, then? | 06:08 |
| *** athomas has quit IRC | 06:08 | |
| lifeless | StevenK: we can look to see if https://review.openstack.org/#/c/81120/ passes | 06:09 |
| lifeless | StevenK: if it passes, I'll +2 and GheRivero or Ng can +2 + +A it | 06:09 |
| StevenK | lifeless: Sounds like a plan. | 06:10 |
| *** akuznetsov has quit IRC | 06:13 | |
| *** vkozhukalov_ has joined #tripleo | 06:22 | |
| *** lazy_prince is now known as Killer_prince | 06:24 | |
| *** mrunge has joined #tripleo | 06:29 | |
| openstackgerrit | Gregory Haynes proposed a change to openstack/tripleo-incubator: Explicitly pip install pbr before its installed as dependency https://review.openstack.org/81126 | 06:29 |
| *** matsuhashi has quit IRC | 06:30 | |
| *** xuhaiwei_ has joined #tripleo | 06:32 | |
| *** rpodolyaka1 has quit IRC | 06:35 | |
| Killer_prince | lifeless: i installed qemu-system-arm and i still see failure.. chroot: failed to run command 'sh': Exec format error | 06:36 |
| Killer_prince | i will be posting complete logs soon.. | 06:36 |
| Killer_prince | its failing at ---sudo -E chroot /tmp/image.fYFM3tCj/mnt sh -c 'PATH=$PATH:/usr/local/bin ; dib-run-parts /tmp/in_target.d/pre-install.d'--- | 06:39 |
| Killer_prince | complete log here. http://paste.openstack.org/show/73723/ | 06:44 |
| Killer_prince | can someone pls help me.. | 06:44 |
| *** rpodolyaka1 has joined #tripleo | 06:45 | |
| *** rpodolyaka1 has quit IRC | 06:46 | |
| *** rpodolyaka1 has joined #tripleo | 06:46 | |
| *** CaptTofu has joined #tripleo | 06:46 | |
| *** rpodolyaka1 has quit IRC | 06:46 | |
| *** rpodolyaka1 has joined #tripleo | 06:46 | |
| *** rpodolyaka1 has quit IRC | 06:47 | |
| *** rlandy has joined #tripleo | 06:47 | |
| *** rpodolyaka1 has joined #tripleo | 06:47 | |
| *** matsuhashi has joined #tripleo | 06:49 | |
| lifeless | Killer_prince: right now we're in crunch period leading up to the release, so I suspect you need to debug this one yourself. I suggest making sure you can run arm binaries locally as a first step | 06:50 |
| *** CaptTofu has quit IRC | 06:51 | |
| Killer_prince | lifeless: okay.. i will look into it. for tracking purpose, does it makes sense to report a bug..? | 06:53 |
| lifeless | sure | 06:54 |
| xuhaiwei_ | In icehouse, we can just deploy a stack with role of controller, compute, object storage and block storage with tuskar, is it right? | 06:57 |
| xuhaiwei_ | we can't deploy other role stack with tuskar, right? | 06:58 |
| *** yamahata has joined #tripleo | 06:59 | |
| *** sseago_ has quit IRC | 06:59 | |
| lifeless | xuhaiwei_: tuskar is a moving target, hard to say. | 07:02 |
| lifeless | xuhaiwei_: but tripleo can deploy many more variations | 07:02 |
| xuhaiwei_ | tripleo can deploy almost every service? keystone, neutron etc? | 07:03 |
| lifeless | marun: yes | 07:04 |
| *** noslzzp has quit IRC | 07:06 | |
| *** rpodolyaka1 has quit IRC | 07:07 | |
| *** gcha has quit IRC | 07:07 | |
| *** gcha has joined #tripleo | 07:08 | |
| Killer_prince | ls -l | 07:10 |
| xuhaiwei_ | lifeless: In tripleo, the role is totally configured in the heat template? | 07:10 |
| giulivo | Killer_prince, | 07:11 |
| giulivo | total 12 | 07:11 |
| giulivo | drwxr-xr-x. 3 root root 4096 Mar 17 14:05 . | 07:11 |
| giulivo | drwxr-xr-x. 19 root root 4096 Mar 17 14:38 .. | 07:11 |
| giulivo | drwxr-xr-x. 22 root root 4096 Mar 17 14:19 stack | 07:11 |
| Killer_prince | giulivo: sorry.. i accidently typed "ls -l" on to my irc client.. did not mean to ask directory listing.. :) | 07:12 |
| giulivo | sure, joking about it | 07:12 |
| Killer_prince | ohh.. cool.. | 07:14 |
| lifeless | xuhaiwei_: configuration in heat, implementation in the disk image you build | 07:14 |
| xuhaiwei_ | ok | 07:15 |
| Killer_prince | is there any documentated steps to try tripleo on ubuntu saucy in its current form.. i would like to setup one and get my hands dirty.. | 07:16 |
| lifeless | Killer_prince: devtest | 07:16 |
| Killer_prince | aha.. so all i have to do is pull tripleo-incubator from github and then use devtest from ther to get going.. | 07:17 |
| Killer_prince | let me give it a try.. | 07:17 |
| lifeless | Killer_prince: github? git.openstack.org/openstack/tripleo-incubator | 07:19 |
| Killer_prince | yup.. my mistake.. | 07:20 |
| tchaypo | +g giulivo | 07:27 |
| *** lazy_prince has joined #tripleo | 07:33 | |
| tchaypo | s/+g/+1/ | 07:38 |
| *** dkehn_ has quit IRC | 07:41 | |
| *** dkehn_ has joined #tripleo | 07:41 | |
| *** jcoufal has joined #tripleo | 07:42 | |
| *** jcoufal has quit IRC | 07:42 | |
| *** jcoufal has joined #tripleo | 07:42 | |
| *** jprovazn has joined #tripleo | 07:48 | |
| *** rcarrillocruz has joined #tripleo | 07:52 | |
| *** bogdando has quit IRC | 07:52 | |
| *** e0ne has joined #tripleo | 07:53 | |
| *** jcoufal has quit IRC | 07:57 | |
| *** ramishra has quit IRC | 08:00 | |
| *** jcoufal has joined #tripleo | 08:00 | |
| *** akrivoka has joined #tripleo | 08:08 | |
| *** nati_ueno has joined #tripleo | 08:11 | |
| openstackgerrit | Ladislav Smola proposed a change to openstack/tripleo-image-elements: Setting negative RamWeigher https://review.openstack.org/81200 | 08:11 |
| *** jtomasek has joined #tripleo | 08:13 | |
| *** ramishra has joined #tripleo | 08:13 | |
| *** matty_dubs|gone is now known as matty_dubs | 08:14 | |
| *** e0ne_ has joined #tripleo | 08:16 | |
| *** bogdando has joined #tripleo | 08:16 | |
| GheRivero | morning all | 08:16 |
| *** sseago has joined #tripleo | 08:16 | |
| *** noslzzp has joined #tripleo | 08:17 | |
| *** rlandy_ has joined #tripleo | 08:18 | |
| *** jtomasek has quit IRC | 08:18 | |
| *** jrist has joined #tripleo | 08:19 | |
| *** e0ne has quit IRC | 08:19 | |
| *** lucasagomes has joined #tripleo | 08:20 | |
| *** jprovazn has quit IRC | 08:21 | |
| *** rlandy has quit IRC | 08:21 | |
| *** akuznetsov has joined #tripleo | 08:22 | |
| *** ramishra has quit IRC | 08:23 | |
| *** jistr has joined #tripleo | 08:26 | |
| *** jistr is now known as jistr|mtg | 08:27 | |
| *** rcarrillocruz has left #tripleo | 08:30 | |
| *** ramishra has joined #tripleo | 08:31 | |
| *** rcarrill` has joined #tripleo | 08:33 | |
| *** morganfainberg is now known as morganfainberg_Z | 08:34 | |
| *** jtomasek has joined #tripleo | 08:36 | |
| *** bauzas has joined #tripleo | 08:37 | |
| *** rcarrill` has quit IRC | 08:38 | |
| *** rcarrillocruz has joined #tripleo | 08:38 | |
| xuhaiwei_ | I read the source of undercloud-source.yaml and found this line: image: Ref: undercloudImage, does this mean using the undercloudImage created by the diskimage-builder, but I can't find the undercloudImage element in tripleo-image-elements, how to create this image? | 08:39 |
| *** rcarrillocruz has quit IRC | 08:40 | |
| *** rcarrillocruz has joined #tripleo | 08:40 | |
| xuhaiwei_ | Can anyone answer this question? | 08:42 |
| *** mkerrin1 has quit IRC | 08:43 | |
| *** hashar has joined #tripleo | 08:44 | |
| *** lucasagomes has quit IRC | 08:46 | |
| *** ramishra has quit IRC | 08:47 | |
| *** rcarrillocruz has quit IRC | 08:47 | |
| *** jp_at_hp has joined #tripleo | 08:47 | |
| *** CaptTofu has joined #tripleo | 08:47 | |
| *** rcarrillocruz has joined #tripleo | 08:47 | |
| *** derekh has joined #tripleo | 08:47 | |
| *** mkerrin has joined #tripleo | 08:49 | |
| *** petertoft has joined #tripleo | 08:50 | |
| *** CaptTofu has quit IRC | 08:52 | |
| *** rlandy__ has joined #tripleo | 08:52 | |
| *** rlandy_ has quit IRC | 08:55 | |
| SpamapS | xuhaiwei_: that is a parameter | 08:59 |
| SpamapS | xuhaiwei_: it can be an ID, or an image name | 08:59 |
| * SpamapS should really actually sleep | 08:59 | |
| *** noslzzp has quit IRC | 09:00 | |
| SpamapS | xuhaiwei_: it is passed into heat on stack creation and update | 09:01 |
| xuhaiwei_ | it is not passed to diskimage-builder? | 09:01 |
| SpamapS | xuhaiwei_: no. diskimage-builder has no idea what a heat template is. | 09:01 |
| *** andreaf has joined #tripleo | 09:02 | |
| *** rlandy_ has joined #tripleo | 09:02 | |
| xuhaiwei_ | Do we decide the deploy image in heat template? | 09:02 |
| SpamapS | xuhaiwei_: diskimage-builds whatever elements you ask for, you upload those to glance, then tell Heat to deploy that image as-needed. | 09:02 |
| SpamapS | xuhaiwei_: we decide the deploy image when we make the call to heat stack-create or stack-update | 09:03 |
| xuhaiwei_ | I see, this imge is pointing to the image in glance? | 09:03 |
| SpamapS | xuhaiwei_: and if we change the ID, Heat will use nova's rebuild command to rebuild the machine with the new image ID. | 09:03 |
| *** ramishra has joined #tripleo | 09:03 | |
| *** rlandy__ has quit IRC | 09:04 | |
| xuhaiwei_ | heat uses this id to download image from glance? | 09:04 |
| SpamapS | no | 09:04 |
| SpamapS | Heat passes this ID to Nova | 09:04 |
| xuhaiwei_ | and nova download it from glance? | 09:05 |
| SpamapS | correct | 09:05 |
| SpamapS | xuhaiwei_: https://git.openstack.org/cgit/openstack/tripleo-incubator/tree/scripts/devtest_undercloud.sh#n70 | 09:05 |
| xuhaiwei_ | Ok, thank you | 09:05 |
| * SpamapS really sleeps now | 09:06 | |
| *** rlandy_ has quit IRC | 09:06 | |
| *** noslzzp has joined #tripleo | 09:08 | |
| lifeless | xuhaiwei_: you should really give devtest a go, its the best way we know of to help people really get the system :) | 09:10 |
| xuhaiwei_ | Yes, I want to do it deadly, but my boos wants me to do the investigation of tripleo first, so I have to have a general idea about tripleo, after that I will do devtest | 09:12 |
| xuhaiwei_ | my boss | 09:12 |
| *** nati_ueno has quit IRC | 09:13 | |
| *** rlandy_ has joined #tripleo | 09:21 | |
| Ng | morning | 09:22 |
| *** jtomasek has quit IRC | 09:23 | |
| *** andreaf has quit IRC | 09:23 | |
| GheRivero | morning | 09:28 |
| Ng | hey GheRivero | 09:34 |
| * Ng tidies up his TripleO slides a little, for presenting at FLOSS UK | 09:34 | |
| openstackgerrit | jan grant proposed a change to openstack/tripleo-image-elements: Workaround: easy_install can't install six from wheels. https://review.openstack.org/78727 | 09:35 |
| *** jcoufal has quit IRC | 09:37 | |
| *** jcoufal has joined #tripleo | 09:39 | |
| *** akuznetsov has quit IRC | 09:39 | |
| tchaypo | lo boss | 09:40 |
| Ng | ;) | 09:41 |
| *** jcoufal has quit IRC | 09:41 | |
| tchaypo | ahhhhh | 09:43 |
| *** jcoufal has joined #tripleo | 09:43 | |
| tchaypo | I've just realised why half the keys on one bluetooth keyboard don't work | 09:43 |
| *** akuznetsov has joined #tripleo | 09:43 | |
| tchaypo | and why the numpad on the other keyboard is moving the cursor | 09:43 |
| openstackgerrit | jan grant proposed a change to openstack/tripleo-incubator: Use a pypi mirror for the bootstrap process. https://review.openstack.org/80949 | 09:44 |
| *** akuznetsov has quit IRC | 09:47 | |
| *** nosnos_ has joined #tripleo | 10:02 | |
| *** lucasagomes has joined #tripleo | 10:02 | |
| *** akuznetsov has joined #tripleo | 10:05 | |
| *** nosnos has quit IRC | 10:05 | |
| *** andreaf has joined #tripleo | 10:06 | |
| *** GheRivero has quit IRC | 10:16 | |
| *** jcoufal has quit IRC | 10:16 | |
| *** nati_ueno has joined #tripleo | 10:20 | |
| *** markmc has joined #tripleo | 10:24 | |
| *** matsuhashi has quit IRC | 10:25 | |
| *** GheRivero has joined #tripleo | 10:27 | |
| *** lucasagomes has quit IRC | 10:30 | |
| *** nati_ueno has quit IRC | 10:33 | |
| *** CaptTofu has joined #tripleo | 10:48 | |
| *** jcoufal has joined #tripleo | 10:48 | |
| *** yamahata has quit IRC | 10:49 | |
| *** lucasagomes has joined #tripleo | 10:50 | |
| *** GheRivero has quit IRC | 10:52 | |
| *** CaptTofu has quit IRC | 10:52 | |
| *** GheRivero has joined #tripleo | 10:53 | |
| openstackgerrit | Derek Higgins proposed a change to openstack/tripleo-image-elements: Treat failure to hear back from neutron as non fatal https://review.openstack.org/81224 | 10:53 |
| *** martyntaylor has joined #tripleo | 10:54 | |
| *** GheRivero has quit IRC | 10:58 | |
| *** martyntaylor has quit IRC | 10:59 | |
| *** martyntaylor has joined #tripleo | 11:03 | |
| *** jistr|mtg has quit IRC | 11:05 | |
| *** rlandy__ has joined #tripleo | 11:07 | |
| *** jrist has quit IRC | 11:10 | |
| *** rlandy_ has quit IRC | 11:10 | |
| *** rwsu has quit IRC | 11:13 | |
| *** rwsu has joined #tripleo | 11:14 | |
| *** jprovazn has joined #tripleo | 11:20 | |
| jp_at_hp | Ng: could you take a look at https://review.openstack.org/#/c/80584/ please? | 11:21 |
| Ng | jp_at_hp: sure, fwiw we're a little blocked on an issue with CI (see the topic), but your change lgtm | 11:23 |
| *** jrist has joined #tripleo | 11:24 | |
| *** e0ne_ has quit IRC | 11:26 | |
| *** martyntaylor has quit IRC | 11:27 | |
| openstackgerrit | Derek Higgins proposed a change to openstack-infra/tripleo-ci: Add script to monitor tripleo jenkins jobs https://review.openstack.org/81236 | 11:28 |
| *** nosnos_ has quit IRC | 11:30 | |
| *** nosnos has joined #tripleo | 11:31 | |
| *** CaptTofu has joined #tripleo | 11:33 | |
| *** fandi has quit IRC | 11:35 | |
| *** xuhaiwei_ has quit IRC | 11:43 | |
| jp_at_hp | slagle: it was suggested in https://review.openstack.org/#/c/80644 that we talk about source-repositories and pip... | 11:50 |
| *** e0ne has joined #tripleo | 11:51 | |
| *** GheRivero has joined #tripleo | 11:53 | |
| openstackgerrit | Jon-Paul Sullivan proposed a change to openstack/tripleo-image-elements: Create pip manifest files of installed packages https://review.openstack.org/80644 | 11:54 |
| *** yamahata has joined #tripleo | 11:56 | |
| slagle | jp_at_hp: i don't *think* there was anything related to source-repositories itself | 11:56 |
| slagle | jp_at_hp: it was actually related to adding pip support to install-packages. so that you could specify if you wanted to install a distro package or a pip package | 11:57 |
| slagle | jp_at_hp: but we actually decided to hold off on that for now | 11:57 |
| jp_at_hp | slagle: ok - I'm just trying to make sure that we can repeat previous tripleo builds, which is what my patch is intended to do for the requirements installs for the venvs. | 11:58 |
| slagle | jp_at_hp: what i did do, was add better support for specifying different install types in https://review.openstack.org/#/c/78794/ | 11:58 |
| slagle | jp_at_hp: i think longer term, i could see us having another install type for pip | 11:59 |
| slagle | so, you may have a os-refresh-config-source-install, os-refresh-config-pip-install, os-refresh-config-package-install | 11:59 |
| jp_at_hp | sure - that all makes good sense, but is independent of ensuring that these different install types are repeatable and replayable, as well as auditable | 12:00 |
| slagle | yea, i think what you've done here is good for now | 12:01 |
| *** ramishra has quit IRC | 12:01 | |
| jp_at_hp | so, as different install types are created, they should ensure that they output what they have done in a way that it can be erused to replicate that install at a future date, without relying on consistent external state (such as pypi not updating). I know that is tough, given people submitting to pypi can reuse versions, but that's a whole 'nother can of worms... | 12:02 |
| jp_at_hp | slagle: well thank you. reviews always welcome :D | 12:02 |
| slagle | i actually have another review that adds package install support to os-*-config, so we might end up rebasing on top of each other for a bit :) | 12:02 |
| *** martyntaylor has joined #tripleo | 12:03 | |
| *** ramishra has joined #tripleo | 12:03 | |
| slagle | jp_at_hp: yea, i'd like to do a similar thing for rpm packages actually. basically save output of rpm -qa at the end. then ability to feed that back into another image build so that the same package versions are used across images | 12:04 |
| slagle | say you're building 5 or 6 images for your deployment, you don't want an intrusive nova update to hit right before you build the last one or something | 12:04 |
| jp_at_hp | slagle: Yup - and dpkg the same Obviously we have slightly different base os'es here :D | 12:04 |
| jp_at_hp | do yum repos make all older versions available? | 12:05 |
| *** hashar has quit IRC | 12:05 | |
| slagle | yes, in theory | 12:05 |
| *** CaptTofu has quit IRC | 12:05 | |
| slagle | sometimes old versions are removed like for a legal reason or something | 12:05 |
| *** CaptTofu has joined #tripleo | 12:06 | |
| slagle | when something was made available when it shouldn't have been, etc | 12:06 |
| *** Shrews has quit IRC | 12:07 | |
| *** jp_at_hp has quit IRC | 12:07 | |
| *** Shrews has joined #tripleo | 12:08 | |
| *** yamahata_ has quit IRC | 12:08 | |
| *** jp_at_hp has joined #tripleo | 12:08 | |
| *** yamahata_ has joined #tripleo | 12:08 | |
| *** mrunge has quit IRC | 12:09 | |
| *** weshay has quit IRC | 12:09 | |
| *** CaptTofu has quit IRC | 12:10 | |
| *** Hefeweizen has quit IRC | 12:10 | |
| *** jprovazn has quit IRC | 12:11 | |
| *** lucasagomes has quit IRC | 12:12 | |
| *** tserong has quit IRC | 12:12 | |
| *** Hefeweizen has joined #tripleo | 12:12 | |
| *** dguerri has quit IRC | 12:13 | |
| *** e0ne has quit IRC | 12:13 | |
| *** dguerri has joined #tripleo | 12:13 | |
| *** tserong has joined #tripleo | 12:17 | |
| *** tserong has joined #tripleo | 12:17 | |
| *** mrunge has joined #tripleo | 12:21 | |
| *** weshay has joined #tripleo | 12:22 | |
| Killer_prince | slagle: in case if you have time, can you review https://review.openstack.org/#/c/79873/ and https://review.openstack.org/#/c/79289/ | 12:24 |
| *** morazi has joined #tripleo | 12:26 | |
| slagle | Killer_prince: sure, will get to it in a bit. still warming up my brain cells this morning :) | 12:26 |
| openstackgerrit | Michael Kerrin proposed a change to openstack/diskimage-builder: Add option PYPI_OFFLINE to the pypi element https://review.openstack.org/81247 | 12:27 |
| Killer_prince | slagle: no problem. you had reviewed it last and i just wanted to ensure that the changes properly address your comments. | 12:28 |
| *** vkozhukalov_ has quit IRC | 12:30 | |
| giulivo | guys, I must be doing something wrong with the undercloud ... it looks to me the undercloud is spawned within the seed, but the heat templates takes as power host host-ip, shouldn't that be the seed-ip ? | 12:31 |
| *** hashar has joined #tripleo | 12:34 | |
| openstackgerrit | Derek Higgins proposed a change to openstack/tripleo-image-elements: Treat failure to hear back from neutron as non fatal https://review.openstack.org/81224 | 12:35 |
| derekh | should unblock overcloud ci ^^ while we figure out how to configure neutron with the tenant id, previous version passed CI, patchset 2 is just a comment (and commit message change), so I think we are good to merge without withing on a second CI run | 12:38 |
| *** michchap has quit IRC | 12:38 | |
| *** michchap has joined #tripleo | 12:38 | |
| Killer_prince | i remember someone pointing to tripleo slides (presentation) about its architecture. somehow, i am not able to find it. | 12:39 |
| Killer_prince | if you have the link, can you pass it on to me pls.. | 12:40 |
| giulivo | Killer_prince, maybe this? https://github.com/openstack/tripleo-incubator/tree/master/presentations | 12:41 |
| *** CaptTofu has joined #tripleo | 12:42 | |
| *** e0ne has joined #tripleo | 12:43 | |
| Killer_prince | giulivo: yup.. thats the one i was looking for. thanks.. :) | 12:44 |
| *** CaptTofu has quit IRC | 12:47 | |
| *** jcoufal has quit IRC | 12:47 | |
| *** e0ne has quit IRC | 12:48 | |
| *** vkozhukalov_ has joined #tripleo | 12:50 | |
| *** morazi has quit IRC | 12:51 | |
| * Killer_prince heading home.. will be back soon.. | 12:51 | |
| *** Killer_prince has quit IRC | 12:51 | |
| *** lazy_prince is now known as killer_prince | 12:51 | |
| *** dprince has joined #tripleo | 12:58 | |
| *** lblanchard has joined #tripleo | 13:05 | |
| *** e0ne has joined #tripleo | 13:13 | |
| *** e0ne has quit IRC | 13:13 | |
| *** e0ne has joined #tripleo | 13:13 | |
| *** julim has joined #tripleo | 13:14 | |
| *** ramishra has quit IRC | 13:14 | |
| *** CaptTofu has joined #tripleo | 13:18 | |
| *** bauzas has quit IRC | 13:30 | |
| *** jprovazn has joined #tripleo | 13:32 | |
| *** rdopieralski has joined #tripleo | 13:36 | |
| *** rdopieralski has quit IRC | 13:39 | |
| *** bauzas has joined #tripleo | 13:39 | |
| *** rdopieralski has joined #tripleo | 13:39 | |
| *** rdopieralski has quit IRC | 13:40 | |
| *** rdopieralski has joined #tripleo | 13:40 | |
| *** sbadia has quit IRC | 13:42 | |
| *** e0ne has quit IRC | 13:44 | |
| *** rdopieralski has quit IRC | 13:45 | |
| *** e0ne has joined #tripleo | 13:45 | |
| *** morazi has joined #tripleo | 13:48 | |
| *** jprovazn has quit IRC | 13:49 | |
| *** sbadia has joined #tripleo | 13:49 | |
| *** jp_at_hp has quit IRC | 13:53 | |
| *** sbadia has quit IRC | 13:56 | |
| *** nosnos has quit IRC | 13:56 | |
| *** jtomasek has joined #tripleo | 13:58 | |
| *** jp_at_hp has joined #tripleo | 13:59 | |
| *** sbadia has joined #tripleo | 13:59 | |
| *** jrist has quit IRC | 14:01 | |
| *** sseago has quit IRC | 14:03 | |
| *** jtomasek has quit IRC | 14:04 | |
| jp_at_hp | So in using devtest.sh a lot I was thinking of seeing how much it could be improved. I'd love to see some comments on https://etherpad.openstack.org/p/tripleo-devtest.sh-refactoring-blueprint ... | 14:05 |
| *** martyntaylor has quit IRC | 14:09 | |
| *** jtomasek has joined #tripleo | 14:10 | |
| *** sseago has joined #tripleo | 14:10 | |
| *** mrunge has quit IRC | 14:12 | |
| *** vkozhukalov_ has quit IRC | 14:12 | |
| *** jrist has joined #tripleo | 14:15 | |
| *** martyntaylor has joined #tripleo | 14:15 | |
| *** CaptTofu has quit IRC | 14:19 | |
| derekh | bnemec: I'm rerunning devtest with your fedora-common element, it failed on friday (probably a unrelated problem) | 14:20 |
| bnemec | derekh: Yeah, I think that's the rebase issue I hit yesterday. There was a change that got missed in the previous rebase and is necessary for devtest to run now. | 14:22 |
| bnemec | The current patch set _should_ work, but I haven't tested it with Fedora yet, just RHEL 7. | 14:22 |
| derekh | bnemec: ok, cool | 14:23 |
| *** slagle has quit IRC | 14:25 | |
| *** vkozhukalov_ has joined #tripleo | 14:25 | |
| *** rdopieralski has joined #tripleo | 14:26 | |
| *** slagle has joined #tripleo | 14:27 | |
| derekh | overcloud neutron workaround passed ci https://review.openstack.org/#/c/81224/2 anybody wanna merge it? | 14:28 |
| Ng | derekh: sure | 14:31 |
| openstackgerrit | A change was merged to openstack/tripleo-image-elements: Treat failure to hear back from neutron as non fatal https://review.openstack.org/81224 | 14:32 |
| *** rdopieralski has quit IRC | 14:32 | |
| *** CaptTofu has joined #tripleo | 14:36 | |
| openstackgerrit | Jun Jie Nan proposed a change to openstack/diskimage-builder: Fix locale no such file or directory issue https://review.openstack.org/81274 | 14:39 |
| *** rdopieralski has joined #tripleo | 14:39 | |
| *** lucasagomes has joined #tripleo | 14:40 | |
| derekh | Ng: thanks | 14:41 |
| *** jcoufal has joined #tripleo | 14:41 | |
| Ng | derekh: np | 14:45 |
| Ng | let us hope that we are unbroken now :) | 14:45 |
| Ng | I'm not going to have much opportunity to land stuff this afternoon, I need to pack and get travelling for FLOSS UK, but I'll stick my nose in after dinner and hotelling tonight | 14:45 |
| Ng | and I'm gonna be on the hp booth all day tomorrow, so if there are quiet periods I'll chuck some time into the backlog | 14:46 |
| derekh | Ng: looks good so far ( we got green runs with that patch) http://goodsquishy.com/downloads/tripleo-jobs.html | 14:46 |
| Ng | \o/ | 14:46 |
| *** rdopieralski has quit IRC | 14:48 | |
| *** hashar has quit IRC | 14:55 | |
| *** morazi has quit IRC | 14:55 | |
| *** akuznetsov has quit IRC | 14:56 | |
| *** hashar has joined #tripleo | 15:00 | |
| *** akuznetsov has joined #tripleo | 15:02 | |
| *** blamar has quit IRC | 15:03 | |
| *** mrunge has joined #tripleo | 15:04 | |
| *** lucasagomes has quit IRC | 15:06 | |
| *** jprovazn has joined #tripleo | 15:06 | |
| rcarrillocruz | jeblair, clarkb: hi, regarding the special case treatment for ref-updated event for https://review.openstack.org/#/c/79205/, I put the rationale behind here: http://paste.openstack.org/show/73683/ | 15:11 |
| rcarrillocruz | an alternative could be play with some set operations (putting an OR to the initial chain of ands to treat the ref-updated) , but imho that would be a bit nasty | 15:11 |
| *** jprovazn has quit IRC | 15:13 | |
| *** blamar has joined #tripleo | 15:17 | |
| *** vkozhukalov_ has quit IRC | 15:22 | |
| *** jcoufal has quit IRC | 15:24 | |
| *** jcoufal has joined #tripleo | 15:29 | |
| *** rdopieralski has joined #tripleo | 15:30 | |
| *** akuznetsov has quit IRC | 15:31 | |
| openstackgerrit | Matthew Gilliard proposed a change to openstack/tripleo-incubator: Allow set-source-vars to consume a git manifest https://review.openstack.org/80617 | 15:32 |
| *** akuznetsov has joined #tripleo | 15:32 | |
| *** vkozhukalov_ has joined #tripleo | 15:33 | |
| devananda | dprince: ping re https://review.openstack.org/#/c/80400 -- are you going to post another rev? | 15:36 |
| *** jtomasek has quit IRC | 15:36 | |
| SpamapS | 'morning | 15:36 |
| *** jrist has quit IRC | 15:37 | |
| *** lucasagomes has joined #tripleo | 15:37 | |
| *** yamahata has quit IRC | 15:39 | |
| devananda | g'mornin | 15:39 |
| *** yamahata has joined #tripleo | 15:39 | |
| dprince | devananda: yes, will do. The suggestion is a good one. I'm running what I already did and it seems to work quite well however | 15:39 |
| devananda | dprince: thanks. i like the fix, just want to keep the option consistent | 15:40 |
| *** derekh changes topic to "TripleO | tripleo-cd running preserve-ephemeral WIP patches and https://review.openstack.org/#/c/62042/ | Using OpenStack to deploy OpenStack;meetings Tuesday 1900 UTC in #openstack-meeting-alt" | 15:49 | |
| *** jtomasek has joined #tripleo | 15:52 | |
| *** jrist has joined #tripleo | 15:52 | |
| *** rupsky has joined #tripleo | 15:56 | |
| *** rupsky has quit IRC | 15:58 | |
| openstackgerrit | therese-mchale proposed a change to openstack/tripleo-image-elements: Secure the mysql installation by - removing anonymous users - removing remote root access - removing the test database - Making sure the root password (if supplied) is set for all root accounts As per /usr/bin/mysql_secure_installation https://review.openstack.org/80570 | 15:58 |
| * Ng bails to head off for FLOSS UK, bbl | 16:03 | |
| SpamapS | http://earthquake.usgs.gov/earthquakes/eventpage/ci15477465 ... 4th aftershock since the 4.4 quake yesterday.. basically directly under me.. been an interesting series of "THUMP" shakes.. :-P | 16:05 |
| *** sbadia has quit IRC | 16:05 | |
| SpamapS | hey is our CI working? | 16:05 |
| SpamapS | derekh: topic changed, so CI is flowing again? | 16:05 |
| derekh | SpamapS: yup, we still have a problem but also applied a workaround https://review.openstack.org/#/c/81224/ | 16:06 |
| SpamapS | derekh: mmk | 16:08 |
| *** bauzas has quit IRC | 16:08 | |
| *** newell has joined #tripleo | 16:11 | |
| *** sseago has quit IRC | 16:12 | |
| *** sseago has joined #tripleo | 16:13 | |
| openstackgerrit | James Slagle proposed a change to openstack/tripleo-image-elements: os-*-config package install support https://review.openstack.org/78816 | 16:15 |
| *** rlandy_ has joined #tripleo | 16:15 | |
| slagle | derekh: can you review ^ plz | 16:16 |
| derekh | slagle: will do | 16:16 |
| openstackgerrit | therese-mchale proposed a change to openstack/tripleo-image-elements: Secure the mysql installation https://review.openstack.org/80570 | 16:18 |
| openstackgerrit | Clint "SpamapS" Byrum proposed a change to openstack/tripleo-incubator: Wait for enough hosts to be available in nova https://review.openstack.org/81106 | 16:18 |
| *** rlandy__ has quit IRC | 16:18 | |
| *** jcoufal has quit IRC | 16:19 | |
| *** bauzas has joined #tripleo | 16:20 | |
| *** epim has joined #tripleo | 16:24 | |
| *** lucasagomes has quit IRC | 16:25 | |
| *** UtahDave has joined #tripleo | 16:25 | |
| *** vkozhukalov_ has quit IRC | 16:25 | |
| *** bauzas has quit IRC | 16:27 | |
| *** noslzzp has quit IRC | 16:31 | |
| *** jcoufal has joined #tripleo | 16:34 | |
| *** rdopieralski has quit IRC | 16:38 | |
| *** jrist has quit IRC | 16:38 | |
| *** jtomasek has quit IRC | 16:38 | |
| openstackgerrit | A change was merged to openstack/tripleo-incubator: Ensure a default value for DEVTEST_PERF_COMMENT https://review.openstack.org/80584 | 16:39 |
| *** jcoufal has quit IRC | 16:40 | |
| *** e0ne_ has joined #tripleo | 16:45 | |
| *** sseago has quit IRC | 16:47 | |
| *** e0ne has quit IRC | 16:49 | |
| openstackgerrit | A change was merged to openstack/tripleo-image-elements: Fix flake8 failures https://review.openstack.org/79348 | 16:50 |
| openstackgerrit | A change was merged to openstack/tripleo-image-elements: Add +x to executable files https://review.openstack.org/79349 | 16:50 |
| killer_prince | is there a way that i can force dib to use a local pip mirror (and not cache) when building images.. seems like when running operations using chroot, pip fails to access local mirrors | 16:52 |
| *** jcoufal has joined #tripleo | 16:55 | |
| killer_prince | slagle: were you able to review https://review.openstack.org/#/c/79873/ and https://review.openstack.org/#/c/79289/ | 16:57 |
| *** jcoufal has quit IRC | 16:57 | |
| *** martyntaylor has quit IRC | 16:58 | |
| *** UtahDave has left #tripleo | 17:00 | |
| *** morganfainberg_Z is now known as morganfainberg | 17:01 | |
| killer_prince | nevermind.. i looked at pypi element and i think i know how to use pip mirrors with dib. | 17:04 |
| SpamapS | killer_prince: pypi uses a local pip mirror IIRC | 17:04 |
| SpamapS | I think we need to rename it actualy. :-P | 17:04 |
| killer_prince | SpamapS: yup.. you are right.. | 17:06 |
| *** mrunge has quit IRC | 17:07 | |
| openstackgerrit | Derek Higgins proposed a change to openstack/tripleo-image-elements: Install python-keystoneclient in the swiftclient venv https://review.openstack.org/75932 | 17:11 |
| killer_prince | i am hoping that setting up PYPIURL to local pip mirror shoul work fine... | 17:12 |
| openstackgerrit | Jonathan Brownell proposed a change to openstack/diskimage-builder: Fix "(None)" seed hostname on Debian https://review.openstack.org/77301 | 17:19 |
| *** killer_prince is now known as lazy_prince | 17:26 | |
| SpamapS | killer_prince: it should.. or PIP_INDEX_URL .. one of those | 17:26 |
| *** hashar has quit IRC | 17:27 | |
| *** jtomasek has joined #tripleo | 17:28 | |
| *** mrunge has joined #tripleo | 17:29 | |
| *** andreaf has quit IRC | 17:32 | |
| slagle | SpamapS: i actually got a successful CI run just earlier today on https://review.openstack.org/#/c/78816/ | 17:34 |
| slagle | SpamapS: you ok if I +A that? | 17:34 |
| jang | SpamapS: pypi can consume the PYPI_MIRROR_URL variable | 17:35 |
| jang | we use that to point at a nearby mirror, not a local on-disk one | 17:35 |
| SpamapS | ah ok | 17:35 |
| SpamapS | jang: thanks.. I should probably RTFM before i OMM (open my mouth) ;) | 17:35 |
| jang | had all sorts of problems with the on-disk one versus something delivered over a medium that groks the moral equivalent of "checkspelling on". | 17:36 |
| SpamapS | slagle: but you got that on patch set 3, right? | 17:36 |
| jang | mostly because it transpires that python package names are case insensitive. At least, as far as enough python coders out in the wild are concerned to make life difficult | 17:36 |
| SpamapS | slagle: that said.. we're not doing package install at all in CI, so if you need to +A now.. I think you're justified. | 17:37 |
| * jang installs MySqLdB>='0.my.god' | 17:37 | |
| SpamapS | jang: gah.. case insensitivity in anything that resembles a path makes my left eye twitch .. o_O | 17:38 |
| *** morganfainberg is now known as morganfainberg_Z | 17:38 | |
| slagle | SpamapS: yes, patchset 4 was just a trivial rebase | 17:38 |
| slagle | are we supposed to see another successful CI run after that? | 17:38 |
| jang | SpamapS: yeah, tell me about it. | 17:39 |
| SpamapS | slagle: definitely | 17:39 |
| SpamapS | slagle: trivial rebase just means the _code_ didn't need merging. It doesn't mean the interfaces didn't change in an incompatible way causing death and destruction .. ;) | 17:40 |
| jang | right, that's enough for today. except to ask: not been completely in the loop, but I'm seeing the tripleo CI break (a lot) and complain about failing checks. I take it this is known about and spurious, right? | 17:40 |
| SpamapS | or that auto-merging didn't cause famine.. plague.. and shame.. | 17:40 |
| * SpamapS is substituting generally negative things for FAIL today | 17:40 | |
| slagle | SpamapS: i thought when jenkins tests the change though, it rebases on master | 17:40 |
| jang | try the noun. "failure". it's old-school. | 17:40 |
| SpamapS | jang: it should not be breaking a lot.. we are being careful with _our_ changes to not merge things into master that break our CI. | 17:41 |
| SpamapS | jang: lately we are only broken by upstream breakage | 17:41 |
| jang | ah yeah. the royal "them" :-) | 17:41 |
| SpamapS | slagle: it does.. but patch set 3 was tested before other things merged. | 17:42 |
| SpamapS | slagle: or rather, there is no evidence that 4 and the tests that ran as 3 are the same | 17:42 |
| SpamapS | slagle: it will all be better when zuul does this for us. ;) | 17:42 |
| slagle | SpamapS: true, i see | 17:43 |
| slagle | b/c when I do CI again, there's no telling if something else will merge, thus invalidating that CI run, before this one gets +A'd | 17:43 |
| SpamapS | jang: We're all dying a little bit because we're not in the gate. Hopefully we can re-start the conversation about getting into the gate and stop this nonsense soon. :-/ | 17:43 |
| SpamapS | slagle: Right, so it is mostly just principle that we have a valid set of checks for the current patch. It is not fool proof. | 17:44 |
| SpamapS | slagle: with zuul, you do have at least the guarantee that the thing that passed is the thing that merges. | 17:44 |
| jang | right. have a great day, all. catch up with you soon. | 17:46 |
| Lotus907efi | so yesterday I submitted a bug about easy_install of pbr failing from an internal pypi mirror | 17:53 |
| Lotus907efi | and GregHaynes worked on a fix for this that explicitly did a pre-install of pbr with pip | 17:54 |
| *** sbadia has joined #tripleo | 17:55 | |
| Lotus907efi | however later in the day there was some discussion about my no_proxy environment variable being set to "no_proxy=192.0.2.0/24,15.126.28.128/27" instead of "no_proxy=192.0.2.1,15.126.28.132" | 17:55 |
| *** jp_at_hp has quit IRC | 17:56 | |
| Lotus907efi | i.e. I was using networks and it was suggested that network settings in the no_proxy env vari might not be correct | 17:56 |
| Lotus907efi | that it might have to be explicit to a host rather than a network | 17:57 |
| greghaynes | Yes, I think that came up with your not being able to install pip IIRC | 17:57 |
| Lotus907efi | since then I have tested this by trying to do a ping to a seed vm with the setting "no_proxy=192.0.2.0/24,15.126.28.128/27" which did fail | 17:57 |
| *** morganfainberg_Z is now known as morganfainberg | 17:58 | |
| Lotus907efi | but if I have this "no_proxy=192.0.2.1,15.126.28.132" and try a ping to my seed vm at 192.0.2.1 the ping does succeed | 17:58 |
| *** derekh has quit IRC | 18:00 | |
| greghaynes | Im surprised ping is effected at all by that var | 18:00 |
| Lotus907efi | however I am wondering if I have a functional pypi mirror and a complete ubuntu saucy repo mirror internal to our DC then do I need proxy settings at all? | 18:00 |
| Lotus907efi | i.e. if I have a ubuntu saucy repo mirror and a pypi mirror inside the DC will there be any outside accesses made at al when running devtest.sh ? | 18:01 |
| greghaynes | If you require the proxy for external http access then yes - the cloud images and elements can do their own http req's | 18:01 |
| greghaynes | rather, cloud images require you can http get them | 18:02 |
| Lotus907efi | but what are they requesting? something beyond ubuntu debs or pypi packages? | 18:02 |
| Lotus907efi | I thought the cloud images were built on the seed vm | 18:02 |
| greghaynes | Its possible, in practice I dont know if any do currently but I know theres at least one patch in review which does this | 18:03 |
| Lotus907efi | so it might be possible that instead of building cloud images internally that pre-built cloud images could be pulled from somewhere outside? | 18:03 |
| greghaynes | They are already | 18:04 |
| Lotus907efi | hmmm | 18:04 |
| *** shakamunyi has joined #tripleo | 18:04 | |
| openstackgerrit | A change was merged to openstack/tripleo-image-elements: os-*-config package install support https://review.openstack.org/78816 | 18:04 |
| greghaynes | Additionally elements can http get things like binary packages if they so chose | 18:05 |
| Lotus907efi | so with my internal ubuntu repo and internal pypi proxy, if I unset the http_proxy and https_proxy env's then something will fail? | 18:05 |
| greghaynes | I would be surprised if no | 18:05 |
| Lotus907efi | hmmm | 18:05 |
| Lotus907efi | I would be interested in learning what else I would have to internalize so this could be brought up in a closed off DC | 18:06 |
| greghaynes | Probably a good idea to set the http_proxy for you and then analyze the access logs | 18:06 |
| Lotus907efi | hmmm, yeah could do that | 18:07 |
| jesusaurus | Lotus907efi: if your end goal is to be able to run without the proxies, then you will probably have to iteratively find all the bits and pieces that go out to the web and either internalize them or work around them | 18:07 |
| *** akrivoka has quit IRC | 18:07 | |
| Lotus907efi | yeah, I understand | 18:07 |
| greghaynes | A wild jesusaurus appears | 18:07 |
| * jesusaurus uses Lurk On IRC. its SuperEffective | 18:08 | |
| *** cwolferh has joined #tripleo | 18:09 | |
| Lotus907efi | actually my end goal would be to be able to bring devstack up in a DC that has no outside access | 18:09 |
| *** cwolferh_ has quit IRC | 18:09 | |
| Lotus907efi | but that is a longer term goal | 18:09 |
| greghaynes | s/devstack/devtest ? | 18:10 |
| Lotus907efi | yeah | 18:10 |
| jesusaurus | greghaynes: whats the diff? | 18:10 |
| Lotus907efi | internally here I have heard different names applied to this | 18:10 |
| jesusaurus | greghaynes: is devtest the full tripleo test? | 18:11 |
| greghaynes | Its the full guts of it AIUI | 18:11 |
| greghaynes | our CI is on my list of things to figure out | 18:11 |
| *** jtomasek has quit IRC | 18:13 | |
| openstackgerrit | Ben Nemec proposed a change to openstack/tripleo-image-elements: Add mysql-mariadb-compat element https://review.openstack.org/78034 | 18:16 |
| openstackgerrit | Ben Nemec proposed a change to openstack/tripleo-image-elements: Add cinder-lio element https://review.openstack.org/78463 | 18:16 |
| openstackgerrit | Ben Nemec proposed a change to openstack/tripleo-image-elements: Don't install percona-toolkit if distro is RHEL7 https://review.openstack.org/78032 | 18:16 |
| openstackgerrit | Ben Nemec proposed a change to openstack/tripleo-image-elements: Factor out tgt-specific parts of cinder element https://review.openstack.org/78462 | 18:16 |
| openstackgerrit | Ben Nemec proposed a change to openstack/tripleo-image-elements: Allow os-svc-daemon to specify dir name with -d option https://review.openstack.org/78033 | 18:16 |
| *** shakamunyi has quit IRC | 18:18 | |
| *** mrunge has quit IRC | 18:23 | |
| *** jtomasek has joined #tripleo | 18:24 | |
| *** shakamunyi has joined #tripleo | 18:27 | |
| SpamapS | jesusaurus: our CI is not quite all of devtest unfortunately | 18:30 |
| SpamapS | jesusaurus: devtest as a whole will get you a full end to end TripleO demo setup with vms on the machine it is run from.. | 18:30 |
| SpamapS | jesusaurus: but our CI has to contrive a few things because: reasons. | 18:31 |
| jesusaurus | ah | 18:31 |
| Lotus907efi | nuts, even though I specified a sources.list file on the actual host system that pointed to our internal Ubuntu repo I just noticed that some later point of image building has reverted back to using the external ubuntu repos | 18:35 |
| Lotus907efi | I think I am at the overcloud image build | 18:35 |
| Lotus907efi | so somewhere the sources.list file I put in place on the underlying real server got "lost" | 18:36 |
| Lotus907efi | is there some place I need to make an alteration so my internal Ubuntu repo sources.list gets used all the time when possible? | 18:36 |
| Lotus907efi | perhaps I need a better understanding of how and where these images get built | 18:37 |
| Lotus907efi | obviously the seed VM image is built on the real host where I run devtest.sh from | 18:38 |
| Lotus907efi | is the undercloud image then built on the seed or still on the real underlying host? | 18:38 |
| *** e0ne_ has quit IRC | 18:39 | |
| greghaynes | Lotus907efi: Yep, the ubuntu element, one sec ill find you the doc for that | 18:40 |
| greghaynes | Lotus907efi: http://docs.openstack.org/developer/tripleo-incubator/devtest_variables.html check out point 7 | 18:41 |
| Lotus907efi | ok | 18:42 |
| Lotus907efi | thanks | 18:42 |
| Lotus907efi | ah ha! I did that and it seems to have reverted back during the build somewhere | 18:43 |
| Lotus907efi | I have in my devtest_variables.sh file: export DIB_COMMON_ELEMENTS="stackuser apt-sources use-ephemeral pypi" | 18:44 |
| Lotus907efi | export DIB_APT_SOURCES=/etc/apt/sources.list | 18:44 |
| *** markmc has quit IRC | 18:44 | |
| Lotus907efi | or does the sources.list file have to reside someplace underneath the tripleo cache directories? | 18:45 |
| greghaynes | no, that should work | 18:45 |
| greghaynes | Are you sure youre re-making the images? | 18:45 |
| greghaynes | as in, USE_CACHE=0 | 18:45 |
| Lotus907efi | well I guess I could do a cleanup-env first just to make sure later today | 18:46 |
| Lotus907efi | I don't think I have that set | 18:46 |
| Lotus907efi | does that go in the devtest_variables.sh file too? | 18:46 |
| Lotus907efi | and what does it do exactly? | 18:46 |
| *** jang1 has joined #tripleo | 18:47 | |
| greghaynes | it can, it should default to 0 and it just means if theres a under/over cloud image on disk already it uses that rather than rebuild | 18:47 |
| *** vkozhukalov_ has joined #tripleo | 18:48 | |
| tchaypo | Actually I wouldn't edit devtest_variables.sh. From memory it sources ~/.devtestrc - easier to keep local settings there so you don't have to fight git | 18:49 |
| *** shakamunyi has quit IRC | 18:49 | |
| *** weshay has quit IRC | 18:50 | |
| greghaynes | devtest.sh does | 18:50 |
| greghaynes | but yes | 18:50 |
| greghaynes | much | 18:50 |
| greghaynes | er, much better | 18:50 |
| lifeless | morning | 18:50 |
| greghaynes | O/ | 18:50 |
| lifeless | SpamapS: can you please run the meeting today? C is still home sick... | 18:52 |
| *** e0ne has joined #tripleo | 18:52 | |
| tchaypo | Ah, only devtest.sh reads it? I need to manually source it for other files? That explains some of the behaviour I've been seeing. | 18:52 |
| *** e0ne has quit IRC | 18:53 | |
| tchaypo | s/files/scripts/ | 18:53 |
| greghaynes | Pretty sure, yes | 18:53 |
| lifeless | I beleive StevenK has a patch altering this | 18:53 |
| SpamapS | lifeless: my pleasure | 18:53 |
| *** rpodolyaka1 has joined #tripleo | 18:53 | |
| *** weshay has joined #tripleo | 18:56 | |
| *** jpeeler has quit IRC | 18:56 | |
| *** jpeeler has joined #tripleo | 18:58 | |
| *** jpeeler has quit IRC | 18:58 | |
| *** jpeeler has joined #tripleo | 18:58 | |
| SpamapS | 2 minutes until meeting.. -> #openstack-meeting-alt | 18:59 |
| *** petertoft has quit IRC | 18:59 | |
| *** pblaho has joined #tripleo | 18:59 | |
| tchaypo | Urgh. I just realised that DST works the opposite of how I thought it did, and these meetings are soon going to be at 5am | 18:59 |
| tchaypo | That'll be exciting | 18:59 |
| greghaynes | eek | 18:59 |
| tchaypo | But at least well get Ng back | 19:00 |
| *** rbrady has quit IRC | 19:10 | |
| *** rbrady has joined #tripleo | 19:11 | |
| *** rlandy_ is now known as rlandy | 19:22 | |
| *** jtomasek has quit IRC | 19:25 | |
| tchaypo | jang1: I'm currently running bandersnatch so I can get my own mirror here | 19:32 |
| tchaypo | I've heard it speeds things up a lot | 19:32 |
| jang1 | pypi-mirror is enough, fwiw (assuming all the requirements are actually specified) | 19:39 |
| clarkb | pypi mirror also gets you wheels | 19:40 |
| jang1 | but yes, it's definitely worthwhile having | 19:40 |
| clarkb | for everything which if compile time is hurting you is a big win | 19:40 |
| jang1 | thus far I've only had a few problems with wheels - mostly where a transitive dependency winds up as the target of easy_install. | 19:40 |
| jang1 | on the whole, they're a big speed win. | 19:40 |
| clarkb | jang1: wouldn't that easy_install still hit the mirror tough/ | 19:42 |
| *** nati_ueno has joined #tripleo | 19:43 | |
| jang1 | it tries to, but there are only wheels generated by pypi-mirror | 19:43 |
| jang1 | not tarballs. | 19:43 |
| clarkb | jang1: that is fixed | 19:43 |
| jang1 | is it now? | 19:44 |
| clarkb | pypi-mirror should do both now | 19:44 |
| jang1 | I'll update it and give it another bash. | 19:44 |
| lifeless | clarkb: it does them to different trees | 19:44 |
| lifeless | clarkb: last I saw | 19:44 |
| clarkb | lifeless: yup | 19:44 |
| *** jtomasek has joined #tripleo | 19:44 | |
| lifeless | clarkb: easy install only reads one index | 19:44 |
| clarkb | lifeless: so point pydistutils as the non wheel site | 19:44 |
| clarkb | because it can'tdo weels | 19:44 |
| clarkb | then point pip at wheels with backup of non wheels | 19:44 |
| lifeless | right, but our tooling isn't taught about wheels vs non wheels yet | 19:45 |
| jang1 | that might be make-workable, but it's atrocious | 19:45 |
| lifeless | also easy install is failing hard behind teh HP firewall | 19:45 |
| lifeless | even with the mirror | 19:45 |
| clarkb | lifeless: interesting, do you use pydistutils.cfg behind the firewall? | 19:45 |
| clarkb | jang1: its not that bad | 19:45 |
| clarkb | we do it upstream or did before pip broke us | 19:45 |
| jang1 | is setuptools amenable to patches to bring sanity to easy_install? | 19:46 |
| jang1 | (there's a joke that ends, "yes, but what a beautiful suit" I'm currently reminded of) | 19:46 |
| clarkb | I don't think so easy_install is something we seem to have to live with according to dstufft | 19:46 |
| lifeless | clarkb: yes, see the chats with Lotus907efi yesterday and the bug with a strace in it ;) | 19:47 |
| clarkb | he has a pip branch that monkey patches easy_install with pip or some such | 19:47 |
| clarkb | which is atrocious :) | 19:47 |
| lifeless | clarkb: yes, and very much desired ;) | 19:47 |
| jang1 | in practice, I'm using a pypi-mirror-created mirror that has solely wheels (or occasionally it leaves a tarball behind instead, but never both) and have only had to work around one problem | 19:48 |
| clarkb | lifeless: Lotus907efi I think the simplest fix for that is to preinstall all your deps including transitive oens with pip | 19:48 |
| jang1 | glance/glanceclient -> cryptography -> six | 19:48 |
| clarkb | jang1: if you pull latest pypi-mirror it should get you wheels and tarballs for everyting | 19:49 |
| jang1 | I'll give it a bash tomorrow. | 19:49 |
| greghaynes | clarkb: yes, thats basically the fix we have for now... still dirty | 19:50 |
| jang1 | although if you're talking about the separate download/mirror directories, I already have that | 19:50 |
| clarkb | jang1: no it will mirror both | 19:50 |
| greghaynes | well, installing the ones that blow up with pip before they get resolved as dependencies | 19:50 |
| clarkb | jang1: so you can point pip at the wheel dir and the non wheel root and it will do the right thing | 19:50 |
| Lotus907efi | my .pydistutils.cfg and .pip/pip.conf file all refernce the mirror directory | 19:51 |
| clarkb | easy_install will not as lifeless points out | 19:51 |
| Lotus907efi | nothign I have here mentions any config that points to the download dir | 19:51 |
| lifeless | Lotus907efi: I believe greghaynes has fixes for your issues, no ? | 19:52 |
| greghaynes | We have the pip install fix, yes | 19:52 |
| greghaynes | Havent fixed root cause | 19:52 |
| Lotus907efi | yes, but I am doubting that they were 100% needed due to me having the network range specified in the no_proxy env variable rather than a direct host specification | 19:53 |
| Lotus907efi | I was asking about this earlier today | 19:54 |
| greghaynes | Yep, im unsure how that would cause the problem you were having but at this point I know not to guess at what things easy_install does and doesnt do | 19:54 |
| Lotus907efi | well if I reverted to an earlier code branch BEFORE your fixes and had the no_proxy env vari set correctly I could test it | 19:55 |
| greghaynes | Yes | 19:55 |
| * tchaypo is on the balcony enjoying the moonlight | 19:56 | |
| greghaynes | That patch also allows us to devtest with binary pip mirror now (woo!) | 19:56 |
| greghaynes | so useful reguardless | 19:56 |
| Lotus907efi | ah | 19:56 |
| * tchaypo just noticed the spider on the railing right beside him busily nibbling on a still-squirming bug | 19:56 | |
| jang1 | right, bedtime. night all. | 19:56 |
| SpamapS | tchaypo: "meanwhile, in Australia..." | 19:57 |
| tchaypo | so about easy_install, the impression i got from a talk by someone (richard giles maybe?) at pycon-au last year is that with the distutils and setuptools merging they're very keen to end the silliness of having two incompatible ways to install packages | 19:58 |
| *** markmc has joined #tripleo | 19:59 | |
| greghaynes | We can only hope | 19:59 |
| *** rpodolyaka1 has quit IRC | 20:01 | |
| *** julim has quit IRC | 20:02 | |
| *** pblaho has quit IRC | 20:02 | |
| Lotus907efi | so if I specify various DIB settings like DIB_APT_SOURCES or PYPI_MIRROR_URL in $HOME/.devtestrc does that over-ride settings in the devtest_variables.sh file? | 20:03 |
| lifeless | tchaypo: yes they are, but the way its happening is terrifying | 20:03 |
| tchaypo | Lotus907efi: yes, devtest_variables.sh is careful not to override anything that's already set | 20:03 |
| Lotus907efi | ah, ok | 20:04 |
| lifeless | Lotus907efi: kindof. That file is meant to capture state so you can resume an environment later. Its written by tripleo, which makes it not super safe to edit. | 20:04 |
| Lotus907efi | I will do that from now on instead of altering devtest_variables.sh | 20:04 |
| tchaypo | from the conversation above i believe you will have to manually `source ~/.devtestrc` before you `source devtest_variables.sh`... but lifeless is saying I'm wrong | 20:04 |
| lifeless | Lotus907efi: I suggest having your own rc file that you source, not touching .devtestrc. | 20:04 |
| Lotus907efi | devtest.sh does a source of $HOME/.devtestrc already | 20:05 |
| Lotus907efi | why shouldn't I touch $HOME/.devtestrc ??? | 20:05 |
| Lotus907efi | if it is outside of my tripleo sources tree would something still modify it? | 20:06 |
| lifeless | oh, .devtestrc is not the file I was thinking of | 20:07 |
| lifeless | tripleorc is what we write to | 20:07 |
| lifeless | .devtestrc is fine to edit | 20:07 |
| tchaypo | phew, my dreams are safe | 20:08 |
| lifeless | tchaypo: you want things to hack on ? | 20:08 |
| tchaypo | lifeless: ah, you did see that | 20:09 |
| tchaypo | there's an open bug somewhere i said i'd poke at today | 20:13 |
| jang1 | hang about. | 20:14 |
| jang1 | I've just been pointed at this: https://review.openstack.org/#/c/74579/ | 20:14 |
| jang1 | in particular, that we should be moving _away_ from using wheels. | 20:14 |
| jang1 | ... seriously?! | 20:14 |
| tchaypo | https://bugs.launchpad.net/tripleo/+bug/1290486 - dhcp agent not serving. want to see if i can replicate | 20:15 |
| greghaynes | jang1: that just adds an option it looks like | 20:16 |
| greghaynes | a --no-use-wheels optino | 20:16 |
| slagle | anyone know what the point of tripleo-image-elements/elements/os-apply-config/os-config-applier/.os-config-applier is? | 20:16 |
| tchaypo | jang1: see mordred's comment - he'd prefer if we update the gates to use a recent pip | 20:16 |
| slagle | i assume just so that we can check an "empty" directory into git? | 20:16 |
| greghaynes | infact... it doesnt look like that patch actually *does* anything | 20:17 |
| greghaynes | oh wait, that option is an arg to pip | 20:17 |
| greghaynes | nvm | 20:17 |
| greghaynes | EMORECOFFEE | 20:17 |
| tchaypo | I managed to get coffee ready abotu 20 minutes into the meeting. things started making more sense after that. | 20:18 |
| jang1 | yeah, then mordred said he was persuaded otherwise by clarkb. | 20:18 |
| greghaynes | clarkb: ^ | 20:19 |
| tchaypo | ah, he did too | 20:19 |
| jang1 | yeah, clarkb: what are you _on_, dude?! :-) | 20:19 |
| mordred | I did? | 20:19 |
| jang1 | that's what it says, yeah | 20:19 |
| mordred | wait - so - | 20:19 |
| jang1 | I'm just wondering what I'm missing. seems arse-backwards to me. | 20:20 |
| jang1 | (that might be local idiom) | 20:20 |
| mordred | that's just in pypi-mirror | 20:20 |
| Lotus907efi | OK, I am writing a wiki page in the hpcloud dakar project pages that tells our devtest testers how to reference my USeast pypi mirror | 20:20 |
| mordred | we need BOTH things in the mirror | 20:20 |
| mordred | and if we have the download download wheels, then we don't get tarballs | 20:20 |
| mordred | but since we MAKE wheels in pypi-mirror | 20:20 |
| mordred | from everythign we download | 20:20 |
| jang1 | quite so. | 20:20 |
| mordred | then downloading only tarballs and tehn creating mirrors | 20:20 |
| Lotus907efi | so I guess I need to change it slightly so they are making changes to $HOME/.devtestrc | 20:20 |
| mordred | gets us a complete mirror that can be used from both versions of python | 20:21 |
| Lotus907efi | instead of devtest_variables.sh | 20:21 |
| mordred | s/oython/pip/ | 20:21 |
| mordred | make sense/ | 20:21 |
| mordred | ? | 20:21 |
| jang1 | as long as we get both. I'll update my pypi0-mirror and see if it does the right thing. | 20:21 |
| Lotus907efi | so is that a different process I should follow to create our internal pypi mirror then? | 20:22 |
| jang1 | since clarkb also just claimed that "it [easy_install] is not tha tbad" I'm just worried that long-term exposure to python's packaging brokenness will permanently atrophy my critical faculties :-) | 20:22 |
| tchaypo | Lotus907efi: and don't forget to remind them to manually source ~/.devtestrc (unless they're running devtest.sh which already does this) | 20:22 |
| jang1 | because it really _is_ that bad. | 20:22 |
| Lotus907efi | tchaypo: OK | 20:22 |
| jang1 | bah, I'll look again in the morning. | 20:22 |
| mordred | jang1: it's AWfUL | 20:23 |
| mordred | jang1: when did clarkb claim easy_install is not that bad? | 20:24 |
| mordred | it's terrible | 20:24 |
| mordred | it'sc completely and utterly broken in all ways and never should be used in any circumstances at all ever | 20:24 |
| jang1 | hold onto that thought. hold it close to your heart. | 20:24 |
| jang1 | with that, you might find your baby brother and escape king jareth. | 20:24 |
| *** dprince has quit IRC | 20:27 | |
| *** epim has quit IRC | 20:27 | |
| SpamapS | mordred: can I use it to smash things? | 20:29 |
| mordred | SpamapS: no. it won't get the job done right | 20:29 |
| SpamapS | damnit | 20:30 |
| *** epim has joined #tripleo | 20:31 | |
| *** sseago has joined #tripleo | 20:36 | |
| *** rpodolyaka1 has joined #tripleo | 20:37 | |
| tchaypo | lifeless: but yes, if you have more things to point me at... | 20:37 |
| SpamapS | all, I'm a bit dubious about it, but this wants a second +2 and is ready for a +A: https://review.openstack.org/#/c/68392/8 | 20:38 |
| * SpamapS goes to find lunch | 20:38 | |
| *** rpodolyaka1 has quit IRC | 20:41 | |
| *** rlandy has quit IRC | 20:42 | |
| *** marun has quit IRC | 20:42 | |
| *** marun has joined #tripleo | 20:45 | |
| *** sseago has quit IRC | 20:51 | |
| *** lblanchard has quit IRC | 20:51 | |
| *** akrivoka has joined #tripleo | 20:57 | |
| *** jtomasek has quit IRC | 20:57 | |
| *** epim has quit IRC | 20:57 | |
| clarkb | what what did Ido? | 20:59 |
| clarkb | mordred: I said the work around for it wasn't that bad | 20:59 |
| clarkb | easy_install itself is terribe. writing a config file to make it less terrible isn't | 20:59 |
| clarkb | I mean if writing a config file is terrible I'm sorry I can't help :) | 20:59 |
| mordred | clarkb: I agree. writing a config file is not that bad | 21:00 |
| tchaypo | It'd be handy if this discussion were noted in the review so we didn't have to keep revisiting it here ;) | 21:02 |
| *** epim has joined #tripleo | 21:04 | |
| clarkb | what review? | 21:05 |
| clarkb | I was talking in relation to pypi-mirror wheels which didn't have a review | 21:05 |
| greghaynes | clarkb: I think hes referring to https://review.openstack.org/#/c/74579/ | 21:06 |
| tchaypo | https://review.openstack.org/#/c/74579/1 | 21:06 |
| clarkb | oh don't use that version of pip/virtualenv | 21:06 |
| tchaypo | If I've followed the conversation, it kicked off because mordred notes on there that "Clarkb convinced me this is all good" but didn't remember why | 21:06 |
| clarkb | but it approved and merged so meh | 21:06 |
| clarkb | I don't grok that commit message either | 21:07 |
| clarkb | why does it matter if the gate pip is older? | 21:07 |
| clarkb | oh I see that is on pypi-mirror it all makes sense now we are talking about wto different things | 21:07 |
| clarkb | I am talking about tripleo using pypi-mirror and ya'll are talking about pypi-mirror itself | 21:07 |
| clarkb | right so what do you want me to say on that review? that review is closed and does a thing | 21:08 |
| clarkb | it fixes jang1's issue from way above about only getting wheels | 21:08 |
| greghaynes | Yea, there was also just some confusion about that review were some people thought it meant were not getting wheels, not that were getting both wheels and not wheels | 21:09 |
| clarkb | you get both | 21:09 |
| greghaynes | Yep, all is well now in the world | 21:09 |
| *** jtomasek has joined #tripleo | 21:11 | |
| openstackgerrit | James Slagle proposed a change to openstack/tripleo-image-elements: Remove mostly empty directories https://review.openstack.org/81368 | 21:16 |
| tchaypo | SpamapS: on 81174 (os-refresh-config hook run later) - I'm assuming from context that when you said "JP" you meant Jon-Paul not James Polley | 21:30 |
| openstackgerrit | James Slagle proposed a change to openstack/tripleo-image-elements: Remove mostly empty directories https://review.openstack.org/81368 | 21:31 |
| *** jtomasek has quit IRC | 21:32 | |
| *** rpodolyaka1 has joined #tripleo | 21:37 | |
| openstackgerrit | A change was merged to openstack/tripleo-image-elements: Enable iscsid on openSUSE https://review.openstack.org/73605 | 21:40 |
| openstackgerrit | A change was merged to openstack/tripleo-image-elements: Fix compatibility link creation for mysql.sock on openSUSE https://review.openstack.org/78909 | 21:40 |
| *** rpodolyaka1 has quit IRC | 21:42 | |
| *** e0ne has joined #tripleo | 21:45 | |
| *** akrivoka has quit IRC | 21:55 | |
| *** CaptTofu has quit IRC | 21:57 | |
| *** CaptTofu has joined #tripleo | 21:58 | |
| openstackgerrit | A change was merged to openstack/tripleo-incubator: Wait for enough hosts to be available in nova https://review.openstack.org/81106 | 22:01 |
| *** CaptTofu has quit IRC | 22:02 | |
| lifeless | tchaypo: ok so things to hack on | 22:04 |
| lifeless | tchaypo: full HA is the big thing | 22:04 |
| lifeless | tchaypo: anything tht makes us have a fully HA envirfnoment - apis, network, db, MQ, | 22:04 |
| greghaynes | Someone needs to massage the galera config to not consume all the ram | 22:05 |
| lifeless | tchaypo: scripts that can run on an instance to quiesce it | 22:05 |
| lifeless | greghaynes: all ram all time | 22:05 |
| greghaynes | such ram, very swap | 22:05 |
| lifeless | much slow | 22:05 |
| tchaypo | quiesce in the sense of reverting to being the "passive" part of an active/passive setup? | 22:06 |
| tchaypo | which is probably not just turning things off, but would mean doing the needful poking to make them get updates from the active partner? | 22:06 |
| lifeless | tchaypo: evacuating all the VMs from a hypervisor, for instance | 22:07 |
| lifeless | tchaypo: and calling the nova api to tell it not to scheduler onto the machine | 22:07 |
| tchaypo | when you say "instance" you mean hypervisor? | 22:07 |
| lifeless | tchaypo: and yes, for A/P stuff, going pasive | 22:07 |
| lifeless | I mean for example | 22:07 |
| tchaypo | I was interpreting that as meaning "guest" | 22:08 |
| openstackgerrit | A change was merged to openstack/tripleo-image-elements: Add cluster support for rabbitmq element https://review.openstack.org/68392 | 22:09 |
| openstackgerrit | A change was merged to openstack/tripleo-image-elements: Fix rabbitmq env file name and log path https://review.openstack.org/70236 | 22:09 |
| openstackgerrit | A change was merged to openstack/tripleo-image-elements: Update openstack config files to support rabbitmq cluster https://review.openstack.org/68393 | 22:09 |
| greghaynes | \O/ merge all the rabbitmq things | 22:09 |
| lifeless | booyah | 22:11 |
| openstackgerrit | therese-mchale proposed a change to openstack/tripleo-image-elements: Secure the mysql installation https://review.openstack.org/80570 | 22:13 |
| lifeless | rbrady: more specifically see https://trello.com/b/0jIoMrdo/tripleo in the 'current MVP column' | 22:14 |
| lifeless | bah | 22:14 |
| lifeless | tchaypo: ^ | 22:14 |
| lifeless | tchaypo: thats what we need to move ahead to the next stage | 22:14 |
| lifeless | greghaynes: so where is xtradb at ? | 22:15 |
| tchaypo | ah, and that gives extra context about what "ha" means | 22:15 |
| greghaynes | https://review.openstack.org/#/c/75540/ The outstanding issue is it consumes too much ram in undercloud (and likely overcloud) | 22:15 |
| greghaynes | kind of stuck on that | 22:16 |
| tchaypo | this is the idea that we need to be able to roll upgrades across the undercloud without affecting any services, right? | 22:16 |
| greghaynes | guru advice needed | 22:16 |
| tchaypo | I see that on the "2 control plane nodes" card there's a note that we need to "fix race conditions in db initialisation/migrations etc" - is that where the reliable-bootstrap we were talking about would come in? | 22:17 |
| lifeless | tchaypo: no, we've got a tie breaker approach for now | 22:18 |
| lifeless | tchaypo: which is to do the init on node 0 | 22:18 |
| lifeless | tchaypo: the reliable thing is 'make it better once it works' kindof stuff. | 22:18 |
| lifeless | tchaypo: medium term. | 22:18 |
| SpamapS | lifeless: With split brain, talking about data integrity, I'm not sure we're even crawling until we know we are preventing that. | 22:19 |
| SpamapS | preventing split brain I mean | 22:20 |
| *** e0ne_ has joined #tripleo | 22:21 | |
| lifeless | SpamapS: galera. 3 nodes. rabbit, 3 nodes. | 22:21 |
| Lotus907efi | I just submitted a core infrastructure bug against run-mirror | 22:22 |
| lifeless | greghaynes: do you know where the memory usage in galera is coming from? | 22:22 |
| Lotus907efi | https://bugs.launchpad.net/openstack-ci/+bug/1294381 | 22:22 |
| SpamapS | lifeless: with rabbit, I'm not sure I understand how we're fencing things even with 3, but "ok" :-P | 22:22 |
| * SpamapS is almost ready to try a softwareconfig based overcloud | 22:22 | |
| lifeless | f*ck yeah | 22:23 |
| greghaynes | lifeless: nope, I twiddled with a couple parameters without really knowing much and didnt have any effect so I do think its in the dive deep into debugging arena... was just hoping someone with domain knowledge might know something offhand that might cause it | 22:23 |
| tchaypo | I think the idea is to use an external consensus mechanism on the 3 nodes to detect split-brain and make sure we have precisely one master | 22:23 |
| SpamapS | lifeless: in theory, if this works, we can deprecate _a lot_ of merge.py | 22:24 |
| *** e0ne has quit IRC | 22:24 | |
| lifeless | tchaypo: no, rabbits cluster should do it | 22:24 |
| lifeless | tchaypo: galeras does | 22:24 |
| lifeless | tchaypo: the stuff we've been discussing around consensus is for solving the 'how do we build the cluster in the first place' aspect | 22:25 |
| tchaypo | Lotus907efi: it looks to me as though this is related to https://review.openstack.org/#/c/74579 - from the conversation here my understanding is that the intent of that change was to make pypi-mirror download the .tar.gz and then generate the wheel | 22:25 |
| lifeless | tchaypo: because these cluster softwares have daft bootstrap processes | 22:25 |
| greghaynes | I *really* want to do some real world testing of the rabbit failover, ive yet to hear from anyone who has actually verified the behavior we expect | 22:25 |
| tchaypo | lifeless: right - rabbit/galera can handle it once the cluster is running, the consensus is only needed for the bootstrap step of picking precisely one node to be the initial master for the cluster | 22:26 |
| tchaypo | once the master is up and slaves connect to it we jettison the consensus | 22:26 |
| greghaynes | things like what happens to clients on slaves while they figure out who new master | 22:26 |
| lifeless | bot for bootstrap if the master fails, the deploy will stop adding nodes to the cluster because the master is dead | 22:26 |
| lifeless | if ~master fails then meh | 22:26 |
| lifeless | I don't see a splitbrain scenario, for the 3-node case, that won't fail to come up | 22:27 |
| lifeless | Lotus907efi: is 0de0e10e753d00f9440a7eb9b5699a3e5a167ede in your pypi-mirror tree ? | 22:27 |
| lifeless | mordred: btw there is a lp pypi-mirror project. Its super confusing. | 22:28 |
| mordred | lifeless: I agree | 22:28 |
| lifeless | mordred: did we get the name on pypi itself ? if so perhaps asklp admins to rename things? | 22:29 |
| mordred | lifeless: yup sure did | 22:31 |
| mordred | lifeless: we don't normally create lp projects for every infra thing | 22:31 |
| lifeless | mordred: I realise that, but having a project with *the right name* is evil. | 22:31 |
| mordred | :) | 22:32 |
| SpamapS | lifeless: what gave you the idea that RabbitMQ has any kind of consensus/quorum/anything algorithm? | 22:34 |
| SpamapS | lifeless: it has master/slaves and the docs specifically say that it does not handle split brain. | 22:34 |
| tchaypo | lifeless: I'm just poking my local cloud into life so I can start asking questions to make sure I understand what is meant by "control plane node" | 22:37 |
| tchaypo | I'm pretty sure I know but I'd rather have the thing in front of me to look at while I'm asking questions | 22:37 |
| lifeless | SpamapS: http://rabbitmq.1065348.n5.nabble.com/rabbitmq-cluster-manage-ops-questions-td21442.html | 22:37 |
| greghaynes | Theres also some interesting caveats for master election that im unsure how our app handles, such as resending all messages which havent been fully ackd | 22:37 |
| lifeless | SpamapS: which docs and which version of the clustering approach | 22:37 |
| lifeless | SpamapS: there was a terrible one in the past | 22:37 |
| *** rpodolyaka1 has joined #tripleo | 22:38 | |
| SpamapS | lifeless: http://www.rabbitmq.com/ha.html | 22:38 |
| SpamapS | and.. | 22:39 |
| Lotus907efi | tchaypo: in the case I was erroring out on yesterday it had a whl file for pip but no tar.gz file in the mirror | 22:39 |
| Lotus907efi | so to work around the issue I had to add a tar.gz file for that to the mirror by hand | 22:40 |
| SpamapS | lifeless: hm ok that is the one I was thinking had the "we don't handle that" but it doesn't. SHoot. | 22:41 |
| lifeless | SpamapS: http://www.rabbitmq.com/clustering.html is what we're using AIUI, but not sure abiout the queu config yet - need to read the opesntack ha gude again | 22:41 |
| tchaypo | If I've understood what clarkb was saying above, that fix means it will now get the .tar.gz and generate a wheel; previously it was just fetching the wheel | 22:41 |
| *** jang1 has quit IRC | 22:41 | |
| Lotus907efi | and lifeless asked me to submit a bug against core irfrastructure for this problem | 22:41 |
| Lotus907efi | hmmm | 22:42 |
| lifeless | Lotus907efi: yep | 22:42 |
| lifeless | Lotus907efi: now we need to see if the code you're running has he fix, or the fix is broken, or $what | 22:42 |
| Lotus907efi | hmm, ok how do I check that? | 22:42 |
| Lotus907efi | (probably $what) :D | 22:42 |
| *** rpodolyaka1 has quit IRC | 22:43 | |
| lifeless | Lotus907efi: the question I asked you up a bit about whether you have that git commit in your pyp-mirror tree is a good start | 22:45 |
| Lotus907efi | hmm, sorry I did not see that | 22:46 |
| Lotus907efi | how do I tell if "Lotus907efi: is 0de0e10e753d00f9440a7eb9b5699a3e5a167ede in your pypi-mirror tree ?" ??? | 22:47 |
| Lotus907efi | where/how do I look for that | 22:47 |
| lifeless | how did you install pypi-mirror? | 22:47 |
| * Lotus907efi is not a git expert by any stretch of the imagination | 22:47 | |
| Lotus907efi | I followed the directions in the README.md file | 22:48 |
| lifeless | which README.md file? | 22:48 |
| lifeless | assume I have -no- context for yourissue | 22:48 |
| openstackgerrit | therese-mchale proposed a change to openstack/tripleo-image-elements: Make MySQL root password configurable https://review.openstack.org/81392 | 22:49 |
| Lotus907efi | https://github.com/openstack/diskimage-builder/blob/master/elements/pypi/README.md | 22:49 |
| Lotus907efi | it says there: pip install git+https://git.openstack.org/openstack-infra/pypi-mirror | 22:49 |
| lifeless | ok | 22:50 |
| lifeless | uhm | 22:50 |
| lifeless | do a pip freeze | 22:50 |
| lifeless | whats the output | 22:50 |
| tchaypo | `pip freeze | grep pypi-mirror` might be more useful than the reams of output | 22:51 |
| lifeless | ah yes :) | 22:51 |
| Lotus907efi | ok | 22:51 |
| Lotus907efi | pypi-mirror==1.0.0.8.g0de0e10 | 22:52 |
| lifeless | ok | 22:52 |
| lifeless | thats the commit that should prevent the issue you had | 22:52 |
| Lotus907efi | hmmm | 22:52 |
| lifeless | from that we can conclude that the fix does not work :) | 22:53 |
| lifeless | or | 22:53 |
| Lotus907efi | so it is $what then? :) | 22:53 |
| clarkb | lifeless: or the install is broken | 22:53 |
| clarkb | lifeless: which does happen with pip | 22:53 |
| lifeless | clarkb: install of pip ? | 22:53 |
| clarkb | lifeless: install of pypi-mirror. | 22:53 |
| lifeless | clarkb: how would it be usable at a then ? | 22:53 |
| clarkb | lifeless: we have seen it where pip installs a thing, says it is installed but doesn't update your executables so you run the old code | 22:54 |
| clarkb | lifeless: if there is an older install | 22:54 |
| clarkb | I am not sayin gthat is happening here. Just pointing it out as a possibility | 22:54 |
| lifeless | Lotus907efi: was there an older install of this? | 22:54 |
| Lotus907efi | this was a fresh system I built just the other day | 22:54 |
| Lotus907efi | I built it specifically for the purpose of being a pypi mirror | 22:54 |
| lifeless | ok so not that | 22:55 |
| lifeless | httppretty also fails to mirror | 22:55 |
| Lotus907efi | yup | 22:55 |
| clarkb | lifeless: we do know that that fix fixed the problem for upstream | 22:55 |
| lifeless | perhaps we should fix that and then try again | 22:55 |
| clarkb | lifeless: but we run older pip | 22:55 |
| lifeless | whats the bug # for that ? | 22:55 |
| lifeless | clarkb: what pip version do you run? Lotus907efi what pip version is installed? | 22:55 |
| clarkb | lifeless: 1.4.1 | 22:55 |
| Lotus907efi | # pip --version | 22:56 |
| Lotus907efi | pip 1.4.1 from /usr/lib/python2.7/dist-packages (python 2.7) | 22:56 |
| *** vkozhukalov_ has quit IRC | 22:56 | |
| Lotus907efi | that is the system copy of pip | 22:56 |
| Lotus907efi | on the mirror system | 22:57 |
| lifeless | Lotus907efi: whats the bug # for httppretty? | 22:59 |
| Lotus907efi | let me look | 22:59 |
| Lotus907efi | 1293717 | 23:00 |
| Lotus907efi | Can I practice my ADD-ness and bring up a different topic as well? | 23:05 |
| Lotus907efi | so I am looking also looking at 01-set-ubuntu-mirror in diskimage-builder because of what I seen where the ubuntu repo being used in my build earlier today switched from our internal mirror to archive.ubuntu.com about 3/4 of the way through the run (about time overcloud images are being built I think) | 23:06 |
| *** nati_ueno has quit IRC | 23:07 | |
| Lotus907efi | I see in 01-set-ubuntu-mirror it is looking for a setting for: DIB_DISTRIBUTION_MIRROR | 23:07 |
| Lotus907efi | How does this relate to setting DIB_APT_SOURCES in devtest_variables.sh to point at a sources.list file that points at a different repo as well? | 23:08 |
| Lotus907efi | I do not see a setting for DIB_DISTRIBUTION_MIRROR in devtest_variables.sh at all | 23:08 |
| *** epim has quit IRC | 23:08 | |
| tchaypo | I think DIB_DISTRIBUTION_MIRROR is something you're expected to set locally if it matters to you | 23:08 |
| Lotus907efi | I started looking at the 01-set-ubuntu-mirror script because in my output log of the run I noticed that was mentioned just before the switch to the external Ubuntu archives | 23:09 |
| Lotus907efi | is setting DIB_DISTRIBUTION_MIRROR mentioned anywhere in some documentation by chance? | 23:10 |
| tchaypo | I've got a note about it in https://etherpad.openstack.org/p/tripleo-newdev-notes | 23:10 |
| tchaypo | but I'm not sure where that came from - maybe it was something StevenK mentioned to me on irc? | 23:10 |
| Lotus907efi | i.e. why would I need to set that if I am already supplying the pointer to my altered sources.list file which only references my internal repo mirror? | 23:10 |
| tchaypo | my guess is that DIB_DISTRIBUTION_MIRROR controls what diskimage-builder does while it's building the image in a chroot, but the sources.list file gets dumped onto the image and used once the vm comes up | 23:12 |
| tchaypo | but that's purely a guess | 23:12 |
| Lotus907efi | ah, ok | 23:13 |
| Lotus907efi | so I will try setting DIB_DISTRIBUTION_MIRROR in my $HOME/.devtestrc and see what happens | 23:14 |
| greghaynes | they should both work, if not docs are wrong | 23:15 |
| greghaynes | the the sources.list method is to provide more flexibility the the DISTRIBUTION_MIRROR can | 23:15 |
| Lotus907efi | yeah, but I did the sources.list thing and still at some points in the run it fetched deb packages from outside instead of from my local repo | 23:16 |
| tchaypo | ah, right. DIB_DISTRIBUTION_MIRROR lets me point at my local ubuntu repo, but sources.list lets me add in some PPAs as well? | 23:16 |
| Lotus907efi | hmmm | 23:17 |
| greghaynes | yep, among other things | 23:17 |
| *** e0ne_ has quit IRC | 23:17 | |
| tchaypo | I'm thinking that if the outside sources are still availabble, apt-get might prefer them at some points | 23:17 |
| greghaynes | A log of where it does that would be useful... sounds like a bug if thats true | 23:17 |
| tchaypo | eg, if they have a newer version of a package | 23:17 |
| lifeless | Lotus907efi: so the ubuntu element supports DIB_DISTRIBUTION_MIRROR. If you need more control the apt-sources element lets you specify exactly what goes in sources.list | 23:19 |
| *** e0ne has joined #tripleo | 23:19 | |
| lifeless | Lotus907efi: DIB_DISTRIBUTION_MIRROR is documented diskimage-builder/elements/ubuntu/README.md | 23:19 |
| Lotus907efi | hmmm | 23:19 |
| Lotus907efi | I did specify the apt-sources element | 23:19 |
| tchaypo | lifeless: so when the card says to run "2 control plane nodes" - the "control plane node" is the single undercloud notCompute0 node, right? | 23:20 |
| greghaynes | I think s/2/3 | 23:20 |
| tchaypo | lifeless: and the thing we need to do is expand undercloud-vm-source.yaml to define another node called notCompute1 and bring up all the services on that node as well? | 23:21 |
| tchaypo | https://trello.com/c/6NuTjWjl/77-run-2-control-plane-nodes definitely says 2, but 3 seems more gooder to me for no particular reason i can identify | 23:21 |
| greghaynes | well, AIUI its really enough notcompute nodes so we can no downtime upgrade, which ends up being odd number > 3 for galera | 23:22 |
| greghaynes | er >= 3 | 23:22 |
| tchaypo | >3 or >=3? | 23:22 |
| tchaypo | if we move to a consensus-based reliable bootstrap, that's going to neccessitate >=3 in that phase as well | 23:23 |
| tchaypo | how does galera behave if n=2? | 23:23 |
| greghaynes | I would imagine not much better than n=1 | 23:23 |
| tchaypo | I'm guessing that when one node goes away the other node stops processing because it can't know what the other is doing? | 23:23 |
| tchaypo | I'm thinking that if I want to tackle this, the first thing I need to do is learn about heat templates and figure out how to expand the template to have another node or two | 23:24 |
| greghaynes | Also re the heat templates, AIUI thats the step once we get enough ha bits in place (rabbit and galera) | 23:24 |
| tchaypo | then there'll have to be changes somewhere to make the services on the other nodes come up as slaves rather than setting themselves up as master | 23:25 |
| lifeless | tchaypo: yes, 3 not 2. but 2 > 1 :) | 23:25 |
| lifeless | greghaynes: so how does one reproduce the galera memory issue? | 23:26 |
| lifeless | greghaynes: just build a seed with your patch? | 23:26 |
| greghaynes | devtest with that patch, it happens on undercloud. For me it OOM kills at will | 23:26 |
| tchaypo | *edits card to say 3* | 23:26 |
| greghaynes | ended up with dead sshd, lots of fun | 23:26 |
| greghaynes | I should say - it happens to the undercloud | 23:27 |
| Lotus907efi | I see the following set in devtest_variables.sh: export NODE_DIST=${NODE_DIST:-"$TRIPLEO_OS_DISTRO"} #nodocs | 23:27 |
| Lotus907efi | where does $TRIPLEO_OS_DISTRO get set? | 23:27 |
| lifeless | _variables | 23:29 |
| tchaypo | Lotus907efi: _variables sources set-os-type | 23:30 |
| Lotus907efi | nope, not here. I grepped for that and that is the only line that showed up | 23:30 |
| Lotus907efi | oh | 23:30 |
| Lotus907efi | ok | 23:30 |
| tchaypo | odd. I grepped and found about 6 references in set-os-type as well as _variables and a few others | 23:31 |
| *** newell has quit IRC | 23:31 | |
| tchaypo | "grep TRIPLEO_OS_DISTRO" in tripleo-incubator/scripts is what I used | 23:31 |
| Lotus907efi | oh | 23:32 |
| Lotus907efi | I just looked in devtest_variables.sh | 23:32 |
| *** morazi has joined #tripleo | 23:32 | |
| tchaypo | and once again we're back at https://etherpad.openstack.org/p/tripleo-incubator-rationalise-ui - it's hard to figure out where a setting comes from when it's set as a side-effect of running some other script | 23:34 |
| Lotus907efi | OK, so set-os-type sets TRIPLEO_OS_DISTRO automagically by looking around at system files but diskimage-builder/elements/apt-sources/README.md talks about manually setting NODE_DIST | 23:35 |
| Lotus907efi | instead of picking up NODE_DIST from what is automagically set in TRIPLEO_OS_DISTRO | 23:36 |
| Lotus907efi | tchaypo: true that | 23:36 |
| lifeless | Lotus907efi: right, because you may want to build fedora images. | 23:39 |
| lifeless | or RHEL | 23:39 |
| lifeless | and what you have installed on your build box is orthogonal. | 23:39 |
| Lotus907efi | ah, ok | 23:39 |
| * Lotus907efi never heard of an "orthogonal" distro :) | 23:40 | |
| tchaypo | *eyeroll* | 23:40 |
| *** e0ne has quit IRC | 23:40 | |
| Lotus907efi | perhaps it is an evolution of Yggdrasil | 23:41 |
| *** sseago has joined #tripleo | 23:41 | |
| tchaypo | this is why hipchat's animated gif support is so nice | 23:41 |
| tchaypo | one cantruly express one's appreciation of a terrible joke with an animated gif of some rolling eyes | 23:41 |
| Lotus907efi | true | 23:41 |
| tchaypo | instead of a lame ascii *eyeroll* | 23:42 |
| lifeless | clarkb: so https://bugs.launchpad.net/openstack-ci/+bug/1293717 | 23:42 |
| greghaynes | For a very brief time I used an irc client that would auto display gifs | 23:42 |
| lifeless | clarkb: thoughts? | 23:42 |
| clarkb | lifeless: see comment #4, the package is broken | 23:42 |
| lifeless | how can we turn off the wheel feature? | 23:43 |
| lifeless | seems to be unbaked as yet | 23:43 |
| clarkb | pypi-mirror doesn't currently support that, but it shouldn't be hard to add that as an option as the wheel step is a distinct part of the pypi-mirror install everything process | 23:43 |
| lifeless | clarkb: oh, so it mirrors everything, then wheels everything ? | 23:44 |
| clarkb | lifeless: yup | 23:44 |
| lifeless | clarkb: so the wheelification failure of http pretty cannot influence mirror correctness for the source mirror | 23:44 |
| clarkb | I don't think so | 23:44 |
| Lotus907efi | so then why did my mirror have a whl file for pip-1.5.4 but no tar.gz file???? | 23:45 |
| clarkb | which is what I meant about pypi-mirror handling this gracefully. It should continue to work | 23:45 |
| clarkb | and if it doesn't then we should fix that | 23:45 |
| clarkb | Lotus907efi: pip != httpretty | 23:45 |
| Lotus907efi | yeah, I know | 23:45 |
| Lotus907efi | just curious | 23:45 |
| clarkb | so two different things here | 23:45 |
| clarkb | lets not mix them and get confused | 23:45 |
| * Lotus907efi has ADD kicking in again | 23:45 | |
| tchaypo | clarkb: Am I right in my understanding that this is a recent change - prior to the change we talked about earlier, pypi-mirror would just download the wheel from upstream if one was available? | 23:47 |
| clarkb | tchaypo: yes prior to that change it had the bad behavior of only grabbing a wheel if it was available | 23:48 |
| Lotus907efi | ummm, when did this change to pypi-mirror get merged? | 23:49 |
| *** xuhaiwei has joined #tripleo | 23:49 | |
| clarkb | Lotus907efi: february 24th according to gerrit | 23:50 |
| Lotus907efi | hmmm, so well before I pulled a copy for here | 23:50 |
| clarkb | Lotus907efi: so did httpretty end up with a tarball? | 23:51 |
| clarkb | it shouldn't havea wheel because the wheel build broke | 23:52 |
| clarkb | if it didnt' end up with a tarball then we should fix that | 23:52 |
| Lotus907efi | yes, I believe it did | 23:52 |
| clarkb | ok so that is working | 23:52 |
| Lotus907efi | let me look to double check | 23:52 |
| Lotus907efi | yup, tar file but no wheel file | 23:53 |
| Lotus907efi | ok, I still have to ask: why did my mirror have a whl file for pip-1.5.4 but no tar.gz file? | 23:54 |
| clarkb | perfect that is what we want | 23:54 |
| clarkb | Lotus907efi: getting to that | 23:54 |
| Lotus907efi | ok | 23:54 |
| clarkb | so can we agree that httpretty is an upstream thing and pypi-mirror handles it just fine? no one will be made if I mark that bug invalid? | 23:54 |
| Lotus907efi | probably, I agree anyway | 23:54 |
| clarkb | now for pip. pypi-mirror special cases pip and a few other things because it depends on them. https://git.openstack.org/cgit/openstack-infra/pypi-mirror/tree/pypi_mirror/cmd/run_mirror.py#n257 | 23:55 |
| clarkb | in particular I think that upgrade-format may allow wheels when maybe it shouldned | 23:55 |
| lifeless | Lotus907efi: when that httpretty error happens | 23:56 |
| lifeless | Lotus907efi: does pypi-mirror keep going? | 23:56 |
| lifeless | Lotus907efi: and does it exit non-zero at the end? | 23:56 |
| clarkb | yup I think we may want a --no-use-wheel added to upgrade_format | 23:57 |
| clarkb | then the wheel stuff a few lines lower will grab wheels for those deps | 23:57 |
| tchaypo | *headdesk* | 23:57 |
| tchaypo | Mar 18 22:51:17 undercloud-undercloud-jd6lsktlel4r kernel: [ 838.373926] ovs-vswitchd[2213]: segfault at 3 ip b7347231 sp bf9d5db0 error 4 in libgcc_s.so.1[b7333000+1b000] | 23:57 |
| lifeless | greghaynes: 18 bytes (18 B) copied, 2.1611e-05 s, 833 kB/s | 23:57 |
| lifeless | Caching xtradb-cluster tarball from http://www.percona.com/redir/downloads/TESTING/Percona-XtraDB-Cluster/5.5.33-23.7.5/release-5.5.33/476/binary/linux/i386/Percona-XtraDB-Cluster-5.5.33-23.7.5.476-static-openssl-1.0.1e.Linux.i386.tar.gz in /home/robertc/.cache/image-create/source-repositories/xtradb_cluster_0a1a511486c77655dcdc45b84917fd2fe7092392 | 23:57 |
| lifeless | % Total % Received % Xferd Average Speed Time Time Time Current | 23:57 |
| lifeless | Dload Upload Total Spent Left Speed | 23:57 |
| lifeless | 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 | 23:57 |
| lifeless | 0 0 0 21998 0 0 13693 0 --:--:-- 0:00:01 --:--:-- 132k | 23:57 |
| lifeless | Server returned an unexpected response code. [404] | 23:57 |
| tchaypo | that'd be why 2/3rds of my overcloud failed to get on the network, i think | 23:57 |
| Lotus907efi | umm, I am fairly sure it keeps going but I don't know about the exit status at the end | 23:57 |
| lifeless | tchaypo: \o/ | 23:58 |
| lifeless | tchaypo: we really can't have nice things can we | 23:58 |
| *** CaptTofu has joined #tripleo | 23:58 | |
| greghaynes | lifeless: ugh, I bet thats because I x86_64 and missed a bug | 23:58 |
| greghaynes | kinda also is a :( for our CI | 23:58 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!