*** galstrom is now known as galstrom_zzz | 00:03 | |
*** darrenc has quit IRC | 00:07 | |
*** darrenc has joined #openstack-ansible | 00:07 | |
*** mcarden has quit IRC | 00:07 | |
*** mcarden has joined #openstack-ansible | 00:09 | |
*** rstarmer has quit IRC | 00:27 | |
*** perniciouscaffei has quit IRC | 00:39 | |
*** perniciouscaffei has joined #openstack-ansible | 00:40 | |
*** phschwartz has quit IRC | 00:47 | |
*** phschwartz has joined #openstack-ansible | 00:49 | |
*** thorst has joined #openstack-ansible | 00:55 | |
*** thorst has quit IRC | 01:04 | |
*** johnmilton has joined #openstack-ansible | 01:08 | |
*** AnarchyAo has joined #openstack-ansible | 01:22 | |
*** weezS has joined #openstack-ansible | 01:42 | |
*** bfg9000 has quit IRC | 01:49 | |
*** dkehn has quit IRC | 01:56 | |
*** thorst has joined #openstack-ansible | 02:03 | |
*** messy has quit IRC | 02:05 | |
*** dkehn has joined #openstack-ansible | 02:08 | |
*** thorst has quit IRC | 02:10 | |
*** johnmilton has quit IRC | 02:12 | |
*** messy has joined #openstack-ansible | 02:17 | |
*** sdake has quit IRC | 02:29 | |
*** johnmilton has joined #openstack-ansible | 02:34 | |
*** weezS has quit IRC | 02:50 | |
*** weezS has joined #openstack-ansible | 02:55 | |
*** weezS has quit IRC | 02:59 | |
*** thorst has joined #openstack-ansible | 03:06 | |
*** messy has quit IRC | 03:14 | |
*** thorst has quit IRC | 03:14 | |
*** sdake has joined #openstack-ansible | 03:21 | |
*** messy has joined #openstack-ansible | 03:23 | |
*** sdake_ has joined #openstack-ansible | 03:38 | |
*** sdake has quit IRC | 03:40 | |
*** jamesdenton has quit IRC | 03:47 | |
*** Mudpuppy has quit IRC | 03:47 | |
*** thorst has joined #openstack-ansible | 04:12 | |
*** Mudpuppy has joined #openstack-ansible | 04:18 | |
*** tek__ has joined #openstack-ansible | 04:18 | |
*** tek__ has quit IRC | 04:18 | |
*** tek__ has joined #openstack-ansible | 04:18 | |
*** thorst has quit IRC | 04:19 | |
*** chhavi has joined #openstack-ansible | 04:21 | |
*** Mudpuppy has quit IRC | 04:23 | |
openstackgerrit | Tuan Luong-Anh proposed openstack/openstack-ansible-ops: Fix typo: remove redundant 'the' https://review.openstack.org/380921 | 04:46 |
---|---|---|
*** tek__ has quit IRC | 04:47 | |
*** tek__ has joined #openstack-ansible | 04:48 | |
*** shausy has joined #openstack-ansible | 04:49 | |
*** tek__ has quit IRC | 04:53 | |
*** gouthamr has quit IRC | 04:56 | |
*** thorst has joined #openstack-ansible | 05:18 | |
*** thorst has quit IRC | 05:24 | |
*** janki has joined #openstack-ansible | 05:36 | |
*** admin0 has joined #openstack-ansible | 05:40 | |
*** admin0 has quit IRC | 05:44 | |
*** drifterza has quit IRC | 05:54 | |
*** tek__ has joined #openstack-ansible | 06:09 | |
*** jorge_munoz has quit IRC | 06:15 | |
*** jorge_munoz has joined #openstack-ansible | 06:17 | |
*** Oku_OS-away is now known as Oku_OS | 06:20 | |
*** sdake has joined #openstack-ansible | 06:22 | |
*** thorst has joined #openstack-ansible | 06:22 | |
*** sdake_ has quit IRC | 06:23 | |
openstackgerrit | Jimmy McCrory proposed openstack/openstack-ansible-os_magnum: Add basic functional tests to role https://review.openstack.org/355155 | 06:27 |
openstackgerrit | Jimmy McCrory proposed openstack/openstack-ansible-os_magnum: Add basic functional tests to role https://review.openstack.org/355155 | 06:29 |
*** thorst has quit IRC | 06:29 | |
*** rgogunskiy has joined #openstack-ansible | 06:49 | |
*** pcaruana has joined #openstack-ansible | 06:53 | |
*** tek__ has quit IRC | 07:04 | |
*** admin0 has joined #openstack-ansible | 07:04 | |
admin0 | good morning | 07:05 |
Mr_Smurf | good morning | 07:05 |
admin0 | smurf :) league of legends ;) ? | 07:05 |
Mr_Smurf | naa, generic blue smurf... | 07:07 |
Mr_Smurf | :) | 07:07 |
admin0 | :) | 07:07 |
admin0 | i am valdating newton + glance/cinder/nova + ceph + 16.04 => getting issues .. searching for someone else who tried this, or who can help | 07:10 |
admin0 | *validating | 07:11 |
*** sdake_ has joined #openstack-ansible | 07:14 | |
*** sdake_ has quit IRC | 07:14 | |
*** sdake has quit IRC | 07:15 | |
*** sdake has joined #openstack-ansible | 07:26 | |
*** thorst has joined #openstack-ansible | 07:27 | |
*** drifterza has joined #openstack-ansible | 07:29 | |
*** thorst has quit IRC | 07:34 | |
*** fops has joined #openstack-ansible | 07:51 | |
*** sdake has quit IRC | 07:58 | |
*** evrardjp has quit IRC | 08:09 | |
*** evrardjp has joined #openstack-ansible | 08:09 | |
*** asettle has joined #openstack-ansible | 08:21 | |
*** jperry has quit IRC | 08:27 | |
*** thorst has joined #openstack-ansible | 08:32 | |
odyssey4me | admin0 this is the purpose of the rc - keep digging to see if you find either a config problem, or a code problem | 08:35 |
*** thorst has quit IRC | 08:40 | |
openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_nova: Update default git branch to stable/newton https://review.openstack.org/380201 | 08:41 |
openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_nova: Update default git branch to stable/newton https://review.openstack.org/380201 | 08:42 |
*** dalees has joined #openstack-ansible | 08:45 | |
pjm6 | good morning guys | 08:50 |
drifterza | o/ | 08:54 |
asettle | odyssey4me: Kelly had finalised her edits for the install guide. It's pretty hefty. You can see both her reports here (attached to the filed bug: https://bugs.launchpad.net/openstack-ansible/+bug/1628958 and https://bugs.launchpad.net/openstack-ansible/+bug/1629798 ) | 08:57 |
openstack | Launchpad bug 1628958 in openstack-ansible "[DOCS] Edits to the OSA install guide" [High,Confirmed] - Assigned to Alexandra Settle (alexandra-settle) | 08:57 |
openstack | Launchpad bug 1629798 in openstack-ansible "[DOCS] Edits to the OSA install guide appendix" [High,Confirmed] - Assigned to Alexandra Settle (alexandra-settle) | 08:57 |
evrardjp | good morning pjm6 drifterza and all the others I forgot | 08:57 |
pjm6 | evrardjp, o/ | 08:58 |
evrardjp | asettle: wehere is the attachment? | 08:58 |
odyssey4me | FYI to anyone who can help dig into this: https://bugs.launchpad.net/openstack-ansible/+bug/1629799 | 08:58 |
openstack | Launchpad bug 1629799 in openstack-ansible "[Newton RC] cinder-volume sometimes not getting installed" [Undecided,New] | 08:59 |
drifterza | evrardjp, could you have a look at my PRs and vote on them please ? | 08:59 |
evrardjp | asettle: on this I mean https://bugs.launchpad.net/openstack-ansible/+bug/1629798 | 08:59 |
openstack | Launchpad bug 1629798 in openstack-ansible "[DOCS] Edits to the OSA install guide appendix" [High,Confirmed] - Assigned to Alexandra Settle (alexandra-settle) | 08:59 |
evrardjp | drifterza: yes sure! | 08:59 |
drifterza | https://review.openstack.org/#/c/374008/, https://review.openstack.org/#/c/375251/ | 08:59 |
drifterza | Thanks dude. | 08:59 |
asettle | DAMNIT, thanks for pointing it out evrardjp | 08:59 |
evrardjp | yw | 09:00 |
asettle | https://bugs.launchpad.net/openstack-ansible/+bug/1629798 | 09:00 |
openstack | Launchpad bug 1629798 in openstack-ansible "[DOCS] Edits to the OSA install guide appendix" [High,Confirmed] - Assigned to Alexandra Settle (alexandra-settle) | 09:00 |
*** chhavi has quit IRC | 09:00 | |
*** chhavi has joined #openstack-ansible | 09:03 | |
openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible-os_cinder: Remove duplicate delegate_to https://review.openstack.org/380992 | 09:07 |
odyssey4me | evrardjp are you looking into https://bugs.launchpad.net/openstack-ansible/+bug/1629799 ? | 09:08 |
openstack | Launchpad bug 1629799 in openstack-ansible "[Newton RC] cinder-volume sometimes not getting installed" [Undecided,New] | 09:08 |
evrardjp | odyssey4me: not yet | 09:08 |
evrardjp | let's say "a little" | 09:09 |
odyssey4me | ok, I'll keep posting findings in the bug for now | 09:09 |
odyssey4me | otherwise we can converse in the channel | 09:09 |
evrardjp | as you should :D | 09:09 |
evrardjp | that's true | 09:09 |
odyssey4me | how does the service restart find the venv? | 09:12 |
evrardjp | odyssey4me: I don't like these kind of messages: Task "Ensure Rabbitmq vhost" has been omitted from the job because the conditional "[u"inventory_hostname == groups['cinder_all'][0]"]" was evaluated as "False" | 09:13 |
odyssey4me | evrardjp I think we could do the rabbit/galera bits with a common set of playbooks in the tests repo | 09:14 |
evrardjp | what do you mean? The full path isn't set? | 09:14 |
odyssey4me | evrardjp not sure - it was just a thought | 09:14 |
odyssey4me | andymccr ^ see comment about rabbitmq/galera - not sure if you have a bit of time to make that happen? | 09:14 |
evrardjp | odyssey4me: even that a little further | 09:14 |
evrardjp | Task "Create DB for service" has been omitted from the job because the conditional "[u"inventory_hostname == groups['cinder_all'][0]"]" was evaluated as "False" | 09:15 |
evrardjp | I will check if the inventory sounds all right | 09:15 |
odyssey4me | evrardjp I'm busy setting up a cinder role test instance. | 09:15 |
andymccr | there is a common db/rabbit test playbook already afaik. | 09:15 |
evrardjp | could you check if you have cinder_all in your inventory group, and not only cinder ? | 09:15 |
evrardjp | odyssey4me: ^ | 09:16 |
evrardjp | and making sure the proper containers are in this group :p | 09:16 |
odyssey4me | evrardjp looks right: https://github.com/openstack/openstack-ansible-os_cinder/blob/master/tests/inventory#L32-L48 | 09:17 |
andymccr | i think we need to remove the static: no on service starts for these services - i've noticed nova has some issues with nova-compute and cinder with cinder-volume on localhost | 09:19 |
evrardjp | interesting | 09:19 |
evrardjp | yes that could be a cause | 09:19 |
andymccr | neutron had a similar problem | 09:19 |
andymccr | you'll notice the service placement task is just ignored for some reason on localhost | 09:19 |
evrardjp | pfff I have these kind of things | 09:19 |
evrardjp | hate* | 09:20 |
andymccr | evrardjp: i genuinely think its a bug in ansible i just cant recreate it in a sensible way to file a bug about it | 09:20 |
andymccr | but the inclusions are so random | 09:20 |
andymccr | i'll put a PR in for nova and cinder | 09:20 |
evrardjp | that's my hating part | 09:20 |
odyssey4me | lemme push up a patch - the dynamic inclusion clearly is very buggy | 09:20 |
andymccr | odyssey4me: we can probably get away with doing it on service includes only | 09:20 |
evrardjp | andymccr: odyssey4me one of you commits, not both :p | 09:20 |
andymccr | if there is a warranted speed benefit | 09:20 |
andymccr | odyssey4me: all yours haha :D | 09:20 |
odyssey4me | 'cos for stable/newton, the dynamic inclusion is only forced for tasks which are static (no inventory things) | 09:21 |
openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_cinder: Remove dynamic inclusion from inventory-based service setup https://review.openstack.org/380999 | 09:22 |
odyssey4me | let's see how that goes - I'll look at the master patch, jmccrory picked up some sort of new package issue | 09:22 |
*** karimb has joined #openstack-ansible | 09:22 | |
*** gtrxcb has joined #openstack-ansible | 09:25 | |
evrardjp | so on the log, every single thing I see skipped and that shouldn't be skipped IMO (like the db sync!) comes from a task that's included with inventory_hostname == groups['cinder_all'][0] | 09:26 |
openstackgerrit | Andy McCrae proposed openstack/openstack-ansible-os_nova: Remove dynamic inclusion from service setup https://review.openstack.org/381003 | 09:26 |
andymccr | evrardjp: which log? | 09:28 |
andymccr | evrardjp: that should only happen because the one host has already failed out and would be cinder-all[0] but since its failed it gets skipped from the include list. | 09:28 |
*** electrofelix has joined #openstack-ansible | 09:28 | |
evrardjp | http://logs.openstack.org/39/380239/4/check/gate-openstack-ansible-os_cinder-ansible-func-ubuntu-trusty/b4e4bf4/console.html#_2016-10-02_04_19_04_627269 | 09:28 |
*** gtrxcb has quit IRC | 09:29 | |
evrardjp | andymccr: well that would make sense, except I see the task target cinder host | 09:29 |
andymccr | but thats ok, db sync should only run once? | 09:29 |
evrardjp | or then it's the complete strategy which is busted | 09:29 |
evrardjp | let me dig deeper, I'll wait for your commit anyway | 09:30 |
evrardjp | well wait is not a proper wording but you get me | 09:30 |
openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-tests: Work around bad pycparser wheel https://review.openstack.org/381007 | 09:31 |
evrardjp | if a host is failed out it should be excluded from the play task, IIRC | 09:31 |
andymccr | evrardjp: it hadnt failed yet - i thought you meant the db sync was never running | 09:31 |
evrardjp | yes it never runs, but a lot of things seems to be skipped for no reason | 09:32 |
andymccr | evrardjp: it is running, just on the cinder host | 09:32 |
odyssey4me | evrardjp i the logs I have, it always runs | 09:32 |
andymccr | you dont want it to run on more than one host - the db is central. | 09:32 |
odyssey4me | the issue is not on the db sync | 09:32 |
evrardjp | well maybe I missed it | 09:33 |
andymccr | its running on the cinder host | 09:33 |
andymccr | it skips localhost cos its only host [0] | 09:33 |
andymccr | it may be better to do "run_once: True" | 09:33 |
evrardjp | I saw it skipped on the cinder host | 09:33 |
evrardjp | let me verify | 09:33 |
andymccr | http://logs.openstack.org/39/380239/4/check/gate-openstack-ansible-os_cinder-ansible-func-ubuntu-trusty/b4e4bf4/console.html#_2016-10-02_04_18_41_140621 | 09:33 |
odyssey4me | andymccr run_once won't work in a role, I don't think | 09:34 |
odyssey4me | it works just fine in a playbook, but not in a role | 09:34 |
odyssey4me | (because the role is executed against only one host without the broader context) | 09:35 |
odyssey4me | for master I'll push up a test review to overhaul the whole way this works to get rid of the inventory references entirely - the review will be a show example for a spec | 09:35 |
odyssey4me | but yeah, for newton we're stuck with this nonsense | 09:36 |
evrardjp | oh well I reread the log, and obviously I misread | 09:36 |
evrardjp | oh you gave me the link | 09:36 |
evrardjp | thanks anyway :) | 09:36 |
*** thorst has joined #openstack-ansible | 09:37 | |
*** sdake has joined #openstack-ansible | 09:44 | |
*** thorst has quit IRC | 09:44 | |
odyssey4me | argh: http://logs.openstack.org/99/380999/1/check/gate-openstack-ansible-os_cinder-ansible-func-ubuntu-trusty/618d3d8/console.html#_2016-10-03_09_36_24_429481 | 09:47 |
odyssey4me | evrardjp andymccr we'll need to get this through asap: https://review.openstack.org/381007 | 09:49 |
evrardjp | odyssey4me: done, you've explained it quite good at the first time, I had no questions for you :p | 09:51 |
andymccr | odyssey4me: will we need a similar PR for nova, since we override that var | 09:51 |
odyssey4me | andymccr shall I add https://github.com/openstack/openstack-ansible-os_nova/blob/master/tests/nova-overrides.yml#L16-L20 to the tests vars? | 09:53 |
odyssey4me | then remove from os_nova | 09:53 |
evrardjp | for sahara too | 09:53 |
odyssey4me | well then remove wherever we have it in the role | 09:53 |
evrardjp | well let me check before saying on sahara and tempest | 09:54 |
andymccr | odyssey4me: wont that mean libvirt-python is installed even when not needed? we could move that into the nova role itself | 09:54 |
andymccr | since it sounds like we may need it | 09:54 |
andymccr | and that way we dont need to override that var for nova | 09:54 |
odyssey4me | andymccr I don't think so - we can check whether it was installed for keystone, but should not have been | 09:55 |
odyssey4me | hmm, pycparser was installed for keystone - but I think that's always the case | 09:57 |
odyssey4me | to make a different mechanism for os_nova and wherever else we'd have to create a new var | 09:57 |
odyssey4me | maybe something like nova_pip_no_binary - like https://github.com/openstack/openstack-ansible-repo_build/blob/master/defaults/main.yml#L64 ? | 09:58 |
odyssey4me | then default it... | 09:58 |
evrardjp | mmm | 09:58 |
andymccr | odyssey4me: ok cool i think it works | 09:58 |
odyssey4me | will you push up a patch, or shall I ? | 09:59 |
odyssey4me | another option we could look into is actually doing a link instead of using the pip package - similar to what we do with ceph? | 09:59 |
evrardjp | odyssey4me: why adding a new variable into the role, where it's supposed to be working fine because it's well handled by the repo_build. We just have to override for tests | 09:59 |
odyssey4me | as suggested in https://bugs.launchpad.net/openstack-ansible/+bug/1625859 | 10:00 |
openstack | Launchpad bug 1625859 in openstack-ansible "Mitaka: libvirt-python package not consistently build in repo " [Undecided,New] | 10:00 |
odyssey4me | andymccr as evrardjp has raised - adding it to the role will mean that even though our repo will have the right package, putting it into the nova role will force it to install from source... that's actually not what we want | 10:01 |
odyssey4me | (it'll force the nova install process to always build from source, ignoring the repo) | 10:02 |
andymccr | hmm | 10:02 |
andymccr | do we want that? | 10:02 |
odyssey4me | I'm kinda thinking that maybe doing the link method will be better | 10:02 |
odyssey4me | the binary and python bits are very clearly tied together | 10:02 |
evrardjp | I like the linking, but what's the point if we fix the problem already with the repo? | 10:03 |
evrardjp | (for this case) | 10:03 |
odyssey4me | evrardjp the point is that the binary installed on compute might not match the binary installed on the repo | 10:03 |
evrardjp | odyssey4me: if they have to be tied, maybe we should pin | 10:03 |
evrardjp | that's what pinned was meant for, right? | 10:04 |
odyssey4me | evrardjp pinned binary? that never works well | 10:04 |
odyssey4me | we tried doing pinned binaries in juno, and it ended badly - ubuntu pulled the pinned package version | 10:04 |
evrardjp | and with the fact that we have a repo that is now an apt-mirror, can't we use that to our advantage? | 10:05 |
*** vnogin has quit IRC | 10:07 | |
*** karimb has quit IRC | 10:08 | |
odyssey4me | evrardjp it's not an apt mirror - just an apt cache | 10:08 |
evrardjp | that's true | 10:08 |
openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-tests: Ensure libvirt-python always compiles from source https://review.openstack.org/381023 | 10:10 |
odyssey4me | for now - I think we should just do ^, then remove the overrides in any roles that have it | 10:10 |
odyssey4me | this makes the workaround similar to how we do it in the repo build | 10:11 |
admin0 | odyssey4me correct .. “ERROR cinder.volume.manager VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: rados and rbd python libraries not found — anyone tested ceph integratio with newton yet” —this looks like a code issue ? some required packages failed to install ? | 10:11 |
odyssey4me | admin0 can you get into the venv and confirm whether the python modules are linked as they should be? | 10:11 |
odyssey4me | admin0 once you've verified that, try restarting the volume service and see whether the error is still there | 10:11 |
*** karimb has joined #openstack-ansible | 10:12 | |
odyssey4me | admin0 this is supposed to link the libraruies: https://github.com/openstack/openstack-ansible-ceph_client/blob/master/tasks/ceph_install_python_libs.yml | 10:12 |
admin0 | the links are there | 10:15 |
odyssey4me | ok, that's good - if you restart cinder-volume then does the error persist? | 10:15 |
openstackgerrit | Merged openstack/openstack-ansible-tests: Work around bad pycparser wheel https://review.openstack.org/381007 | 10:16 |
admin0 | will upload cirros again to test | 10:16 |
admin0 | i meant create another volume again | 10:16 |
*** rgogunskiy has quit IRC | 10:17 | |
*** vnogin has joined #openstack-ansible | 10:17 | |
*** rgogunskiy has joined #openstack-ansible | 10:18 | |
*** Mudpuppy has joined #openstack-ansible | 10:20 | |
admin0 | odyssey4me: https://gist.githubusercontent.com/a1git/6f24e6ffe1863241fddf98a641d599be/raw/7a8b6f61280c10604473890ff27cc4fb1eadec02/gistfile1.txt — volumeID: b84ad51e-f98f-4784-aaa8-ff7a16287b3a | 10:24 |
admin0 | let me gist the configs i used as well | 10:24 |
pjm6 | i just updated the playbooks to the last tag | 10:24 |
pjm6 | 13.3.4 and now i get an error executing "Run venv process script" | 10:25 |
pjm6 | in repo_build of openstack-infraestructure.yml | 10:25 |
*** Mudpuppy has quit IRC | 10:25 | |
pjm6 | anyone had this error ? | 10:25 |
admin0 | odyssey4me: config used: https://gist.github.com/a1git/f55bde9e46f58fb608e6bb87d9c7224c | 10:27 |
admin0 | env.d has a cinder.yml with the is_metal: false override | 10:28 |
pjm6 | I think that the script /opt/op-venv-script.sh is the error | 10:30 |
admin0 | same goes for glance as well ( i am using glance backend to ceph as well) | 10:32 |
evrardjp | odyssey4me: did you see the bad news there? https://review.openstack.org/#/c/380999/1 | 10:33 |
andymccr | evrardjp: thats the PR he put in for the pycparser bit already i believe! | 10:38 |
evrardjp | oh ok | 10:39 |
evrardjp | I'm feeling a little slow today | 10:39 |
andymccr | haha evrardjp have yourself a leffe and all will be right again :D | 10:40 |
evrardjp | that's killing the diseases indeed | 10:40 |
evrardjp | but if I have to chose: https://www.wikiwand.com/en/Tripel_Karmeliet | 10:41 |
openstackgerrit | Andy McCrae proposed openstack/openstack-ansible-tests: Work around bad pycparser wheel https://review.openstack.org/381036 | 10:41 |
andymccr | backport ^ | 10:42 |
*** thorst has joined #openstack-ansible | 10:42 | |
odyssey4me | evrardjp yes, I linked it earlier: http://logs.openstack.org/99/380999/1/check/gate-openstack-ansible-os_cinder-ansible-func-ubuntu-trusty/618d3d8/console.html#_2016-10-03_09_36_24_429481 | 10:46 |
odyssey4me | not sure if they pycparser will help it | 10:46 |
ioni | i have around 130k messages in queue ready, due to failing ceilometer, is there a way to to drop all the messages from rabbitmq? | 10:46 |
openstackgerrit | Andy McCrae proposed openstack/openstack-ansible-tests: Move "--no-binary libvirt-python" to ansible-tests https://review.openstack.org/381043 | 10:47 |
odyssey4me | hmm, I see the pycparser ref - maybe it does | 10:48 |
odyssey4me | ioni sure, I think you can just delete the queue and it'll recreate if you restart the services | 10:48 |
*** openstackgerrit has quit IRC | 10:48 | |
pjm6 | guys it seems that using the stable/mitaka repo solves that error (at least seems to skipped that error) :) | 10:48 |
evrardjp | andymccr: https://review.openstack.org/#/c/381023/1 | 10:49 |
andymccr | ahh | 10:49 |
andymccr | haha | 10:49 |
andymccr | ok i'll remove mine | 10:49 |
*** openstackgerrit has joined #openstack-ansible | 10:49 | |
*** thorst has quit IRC | 10:49 | |
evrardjp | your message was better. | 10:50 |
evrardjp | but whatever! | 10:50 |
odyssey4me | andymccr ok, adding the no-binary flag is harmless: http://logs.openstack.org/23/381023/1/check/gate-openstack-ansible-tests-ansible-func-ubuntu-trusty/032b4b0/console.html#_2016-10-03_10_21_10_410021 | 10:50 |
odyssey4me | it only instals if it's in the requirements | 10:50 |
andymccr | odyssey4me: yeah i tested | 10:51 |
openstackgerrit | Andy McCrae proposed openstack/openstack-ansible-os_nova: Move "--no-binary librit-python" to ansible-tests https://review.openstack.org/381045 | 10:51 |
andymccr | the nova PR ^ | 10:51 |
andymccr | changed the commit message to include the correct PR/depends-on tag | 10:51 |
evrardjp | odyssey4me: cool that's what we 'thought then! | 10:51 |
evrardjp | mo librit? | 10:52 |
odyssey4me | andymccr s/librit/libvirt/ | 10:52 |
evrardjp | andymccr: ^ | 10:52 |
odyssey4me | FYI https://bugs.launchpad.net/cinder/+bug/1629726 | 10:52 |
openstack | Launchpad bug 1629726 in OpenStack Compute (nova) "recompiled pycparser 2.14 breaks Cinder db sync and Nova UTs" [Critical,Confirmed] | 10:52 |
openstackgerrit | Andy McCrae proposed openstack/openstack-ansible-os_nova: Move "--no-binary libvirt-python" to ansible-tests https://review.openstack.org/381045 | 10:52 |
andymccr | librit sounds better anyway | 10:52 |
evrardjp | :D | 10:53 |
odyssey4me | :p | 10:53 |
odyssey4me | evrardjp can you up your vote now that the gate's passed? https://review.openstack.org/#/c/381023/ | 10:53 |
openstackgerrit | Donovan Francesco proposed openstack/openstack-ansible: Add integrated Trove deployment. https://review.openstack.org/374008 | 10:53 |
evrardjp | odyssey4me: done | 10:54 |
evrardjp | I'm still not convinced it's the best we could/should do, but I could see it working somehow | 10:54 |
evrardjp | thanks drifterza, I'm on my way to test it btw | 10:54 |
odyssey4me | this is implementing the same sort of changes as done in the integrated build | 10:55 |
drifterza | cool evrardjp | 10:55 |
drifterza | I added the telemetry vhost stuff aswell | 10:55 |
odyssey4me | andymccr os_sahara and os_tempest need the same patch as os_nova | 10:56 |
andymccr | odyssey4me: true, will fix | 10:56 |
openstackgerrit | Andy McCrae proposed openstack/openstack-ansible-os_tempest: Move "--no-binary libvirt-python" to ansible-tests https://review.openstack.org/381048 | 10:58 |
openstackgerrit | Andy McCrae proposed openstack/openstack-ansible-os_tempest: Move "--no-binary libvirt-python" to ansible-tests https://review.openstack.org/381048 | 10:58 |
pjm6 | well didn't work after all :\ using the last stable/mitaka I have errors in repo_build | 10:59 |
pjm6 | "Run venv process script" | 10:59 |
pjm6 | the script bash /opt/op-venv-script.sh returns code is 99 | 10:59 |
andymccr | odyssey4me: sahara isnt using central testing yet so i dont think we can move it over. | 10:59 |
odyssey4me | andymccr hmm, I think there's a review on the go for that - we can just update it | 11:00 |
pjm6 | pastebin.com/MpDNLyMb | 11:00 |
andymccr | odyssey4me: makes sense. | 11:00 |
pjm6 | anyone have a clue about this? | 11:00 |
odyssey4me | andymccr in fact, as I recall, that review needs some of your attention | 11:00 |
odyssey4me | pjm6 looking | 11:01 |
pjm6 | odyssey4me, thanks | 11:01 |
odyssey4me | pjm6 it looks like the same mycparser error might be hitting us there | 11:01 |
odyssey4me | odd, because the u-c should be protecting us | 11:01 |
openstackgerrit | Andy McCrae proposed openstack/openstack-ansible-os_ironic: Adjust networking for flat network to work https://review.openstack.org/381049 | 11:02 |
pjm6 | odyssey4me, how do you figure out that was pycparser ? | 11:02 |
pjm6 | :o | 11:02 |
odyssey4me | oh no - my mistake - the error is at line 1078 | 11:03 |
odyssey4me | Could not find a version that satisfies the requirement kombu<3.1,>=3.0.36 (from celery) (from versions: 3.0.34) | 11:03 |
odyssey4me | it looks like the tag may have gotten caught in the middle of a requirements update set somehow | 11:03 |
odyssey4me | pjm6 can you register a bug for that please? I'll see if I can track it down tomorrow | 11:04 |
pjm6 | oh true sorry, I miss it :\ | 11:04 |
pjm6 | Sure odyssey4me, could you guide me where I can do that? | 11:04 |
pjm6 | in launchpad? | 11:04 |
odyssey4me | pjm6 yes | 11:05 |
odyssey4me | https://bugs.launchpad.net/openstack-ansible/+filebug | 11:05 |
odyssey4me | pjm6 put the error I've just quoted into the bug report, I'll try and replicate it tomorrow and figure out where it's coming from | 11:05 |
odyssey4me | this shouldn't happen | 11:06 |
pjm6 | the error that is in line 1078 right? | 11:06 |
odyssey4me | yep | 11:06 |
pjm6 | well the description is too long | 11:07 |
pjm6 | maybe could do | 11:07 |
pjm6 | Could not find a version that satisfies the requirement kombu<3.1,>=3.0.36 ? | 11:07 |
odyssey4me | the subject can just be Mitaka <tag>: repo build fails to find kombu | 11:08 |
odyssey4me | then in the description you can put the whole error line | 11:08 |
pjm6 | ah so that line is just the Further information? | 11:08 |
odyssey4me | and any custom settings you have relating to the repo/packages | 11:08 |
odyssey4me | you could gist the whole log file so that's it's a permalink - pastebin clears files so a reference to there will not last | 11:09 |
odyssey4me | andymccr this is the sahara fixup to use ansible 2.1.1 and the central tests repo: https://review.openstack.org/353612 | 11:10 |
pjm6 | odyssey4me, done | 11:10 |
pjm6 | i will try to figure out though which change was made | 11:10 |
odyssey4me | pjm6 link? | 11:11 |
andymccr | odyssey4me: yeah im looking at it now - gonna try fix it up! | 11:11 |
odyssey4me | pjm6 it would have been a SHA bump - the changes made upstream | 11:11 |
odyssey4me | pjm6 that's why the head of mitaka is working fine, because it's done another SHA bump | 11:11 |
pjm6 | https://bugs.launchpad.net/openstack-ansible/+bug/1629841 | 11:11 |
openstack | Launchpad bug 1629841 in openstack-ansible "Mitaka stable: repo build fails to find kombu " [Undecided,New] | 11:11 |
odyssey4me | it is weird that it fails the build though... I'll try and replicate it | 11:11 |
odyssey4me | pjm6 oh hang on - you say it's failing in stable/mitaka? | 11:12 |
odyssey4me | pjm6 I thought it was failing on a specific tagged version? | 11:12 |
pjm6 | first I tried in a tag | 11:12 |
*** sdake_ has joined #openstack-ansible | 11:12 | |
pjm6 | after it i tried with the stable/mitaka branch | 11:12 |
odyssey4me | pjm6 so it failed on both? | 11:12 |
pjm6 | yes | 11:12 |
odyssey4me | please add the tag you tried and that you also tried with stable/mitaka | 11:12 |
odyssey4me | to the bug report I mean | 11:13 |
pjm6 | its possible to edit? | 11:13 |
pjm6 | I already find it | 11:13 |
odyssey4me | yep | 11:13 |
*** sdake has quit IRC | 11:14 | |
pjm6 | updated =) | 11:14 |
admin0 | odyssey4me: i can verify that it setup the keyrings correctly and is pointing to the right pool | 11:14 |
odyssey4me | admin0 so the volume service, after a restart, is finding the libs? | 11:15 |
admin0 | “….is finding the libs?” — how do I verify this :D | 11:16 |
pjm6 | odyssey4me, if I rollback to a previous version, would have a problem? as I initiated this | 11:16 |
odyssey4me | admin0 the volume service started without any errors in the log file | 11:16 |
pjm6 | for example 13.3.0 ? | 11:17 |
odyssey4me | pjm6 oh, this is your production environment? | 11:17 |
pjm6 | no, just testing | 11:17 |
odyssey4me | pjm6 try a test build with just one tag back | 11:17 |
odyssey4me | ie 13.3.3 | 11:17 |
pjm6 | I'm trying to use shared storage (NOVA) | 11:17 |
pjm6 | with OCFS2 idk if anyone tried yet (via iSCSI) | 11:18 |
pjm6 | ok will do that | 11:18 |
openstackgerrit | Andy McCrae proposed openstack/openstack-ansible-os_sahara: Ansible 2.1.1 role testing https://review.openstack.org/353612 | 11:19 |
odyssey4me | andymccr evrardjp a review of https://review.openstack.org/380759 would be nice to stabilise os_neutron | 11:19 |
evrardjp | did we check for tempest and sahara for the pip install options btw? | 11:20 |
odyssey4me | evrardjp sahara - andymccr's working on it in https://review.openstack.org/353612 | 11:20 |
andymccr | evrardjp: tempest should be done | 11:20 |
odyssey4me | evrardjp tempest the patch is in: https://review.openstack.org/381048 | 11:20 |
evrardjp | odyssey4me: andymccr's done the review | 11:21 |
andymccr | sahara needs to move to central test - so im working on that, but also doesnt need libvirt in the testing | 11:21 |
andymccr | so i think that one is ok | 11:21 |
evrardjp | ok | 11:21 |
admin0 | odyssey4me: https://gist.githubusercontent.com/a1git/d9740013b66fa775628c27532ccb8da7/raw/3e1d19774318e7818d63b5bfc8315ec120ab0855/gistfile1.txt — looks OK to me but i might not be able to spot issues there | 11:21 |
*** fops has quit IRC | 11:22 | |
odyssey4me | admin0 was that from the cinder-volume container? | 11:23 |
admin0 | yes | 11:23 |
odyssey4me | admin0 if so - looks fine to me, which means the libs were found | 11:23 |
odyssey4me | were there errors before that? | 11:23 |
admin0 | not that i could see | 11:23 |
odyssey4me | ok, then the issue is not related to finding the libs it seems | 11:24 |
odyssey4me | what was the original issue you saw, and where did you see it? | 11:24 |
openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_glance: Use dictionary for service group mappings https://review.openstack.org/378930 | 11:26 |
*** retreved has joined #openstack-ansible | 11:32 | |
openstackgerrit | Merged openstack/openstack-ansible-tests: Ensure libvirt-python always compiles from source https://review.openstack.org/381023 | 11:32 |
*** fops has joined #openstack-ansible | 11:33 | |
*** jrollen has joined #openstack-ansible | 11:35 | |
openstackgerrit | Donovan Francesco proposed openstack/openstack-ansible: Adding Trove Horizon dashboard https://review.openstack.org/375251 | 11:35 |
odyssey4me | drifterza would you be able to divert your attention from trove/monasca for the next week or so to help test and triage issues for the newton release? | 11:36 |
odyssey4me | it would be very helpful to have more testers and bug triage helpers | 11:37 |
drifterza | okay sure | 11:37 |
drifterza | just take from launchpad ? | 11:37 |
odyssey4me | in particular I don't think anyone's been doing any multi node testing with thorough tests after the build, so if you have the lab facilities for that then that would be great | 11:37 |
*** johnmilton has quit IRC | 11:38 | |
*** Mudpuppy has joined #openstack-ansible | 11:38 | |
odyssey4me | drifterza yeah, look out for anything newton specific and help triage using an AIO - and ideally build a multi-node test environment on the side and throw some normal use cases at it to see if things break, then report any bugs you find | 11:38 |
drifterza | k cool | 11:38 |
odyssey4me | if you spot a problem, feel free to report without doing triage - triage can come later | 11:39 |
odyssey4me | the report of the issue brings awareness, and people may get to triage it before you do | 11:39 |
openstackgerrit | Merged openstack/openstack-ansible-tests: Work around bad pycparser wheel https://review.openstack.org/381036 | 11:39 |
*** cbits has joined #openstack-ansible | 11:40 | |
*** gouthamr has joined #openstack-ansible | 11:41 | |
odyssey4me | thanks drifterza | 11:41 |
odyssey4me | also drifterza if you have ceph builds, then as you've seen with admin0's reports it looks like it needs better testing and there may be issues there | 11:42 |
odyssey4me | interesting new occasional test failure in os_neutron: http://logs.openstack.org/59/380759/4/gate/gate-openstack-ansible-os_neutron-ansible-func-ubuntu-trusty/88f62d2/console.html#_2016-10-03_11_42_25_064895 | 11:46 |
*** thorst has joined #openstack-ansible | 11:47 | |
*** sdake_ is now known as sdake | 11:52 | |
openstackgerrit | Andy McCrae proposed openstack/openstack-ansible-os_sahara: Ansible 2.1.1 role testing https://review.openstack.org/353612 | 11:55 |
openstackgerrit | Donovan Francesco proposed openstack/openstack-ansible: Adding Trove Horizon dashboard https://review.openstack.org/375251 | 11:56 |
admin0 | drifterza: my test setup involves 3 controllers, 3 ceph nodes, 3 compute nodes , 1 log server .. first do it all on KVM based lab and then replicate it to hardware | 11:56 |
*** rgogunskiy has quit IRC | 11:58 | |
mgariepy | good morning everyone | 11:58 |
odyssey4me | drifterza admin0 this may be a very useful tool for testing if you have a server for it: https://github.com/openstack/openstack-ansible-ops/tree/master/multi-node-aio | 11:58 |
admin0 | i have multiple servers that can handle this :) | 11:59 |
admin0 | so if I switch to this, and test it all | 11:59 |
admin0 | hmm.. | 11:59 |
*** karimb has quit IRC | 12:00 | |
openstackgerrit | Merged openstack/openstack-ansible-os_cinder: Remove dynamic inclusion from inventory-based service setup https://review.openstack.org/380464 | 12:00 |
*** rgogunskiy has joined #openstack-ansible | 12:01 | |
*** rgogunskiy has quit IRC | 12:01 | |
admin0 | odyssey4me: this does not cover ceph right ? | 12:02 |
*** dragonmaster has joined #openstack-ansible | 12:02 | |
*** johnmilton has joined #openstack-ansible | 12:04 | |
*** johnmilton has quit IRC | 12:04 | |
*** johnmilton has joined #openstack-ansible | 12:05 | |
admin0 | how do I restart all *-api* services from deploy server :) | 12:07 |
admin0 | shortcut ? | 12:07 |
openstackgerrit | Donovan Francesco proposed openstack/openstack-ansible: Add integrated Trove deployment. https://review.openstack.org/374008 | 12:07 |
drifterza | for loop admin0 | 12:08 |
admin0 | :D | 12:08 |
drifterza | don't know of a ansible way. | 12:08 |
drifterza | :P | 12:08 |
admin0 | in the log server, where do the logs end up ? | 12:08 |
odyssey4me | admin0 you could submit a patch to extend it to cover ceph too | 12:09 |
admin0 | iinside the container in /openstack, i only see myself | 12:09 |
admin0 | odyssey4me: i will do that, for that, i need to get the ceph working first :) | 12:09 |
odyssey4me | admin0 for now, perhaps rather work on that while you wait for deployments - but keep testing deployments with your current tooling | 12:10 |
admin0 | i will spin up the multi-node aio in another server | 12:11 |
admin0 | while i do not break this one | 12:11 |
drifterza | anyone else getting : Failed to build posix-spawn ? | 12:11 |
*** psilvad has joined #openstack-ansible | 12:11 | |
openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_cinder: Remove duplicate delegate_to https://review.openstack.org/380992 | 12:11 |
admin0 | neutron failed when i enabled vpnaas, failed on some openswan stuff .. — so its removed from my variables .. | 12:11 |
admin0 | on the logs server, where do the logs end up in | 12:12 |
admin0 | i want to confirm my findings with someone else before opening up a bug report ( so that its not my setup that might be the issue) | 12:13 |
*** v1k0d3n has quit IRC | 12:14 | |
drifterza | cat /etc/rsyslog.conf | 12:14 |
drifterza | inside the log container | 12:14 |
*** hughmFLEXin has joined #openstack-ansible | 12:14 | |
*** Mudpuppy has quit IRC | 12:14 | |
admin0 | "/var/log/log-storage/%hostname%/%programname%.log" == while /var/log/log-storage has no files in it | 12:15 |
odyssey4me | admin0 rather do a bug report - the triage process includes a step for someone else to replicate | 12:15 |
openstackgerrit | Andy McCrae proposed openstack/openstack-ansible-os_sahara: Ansible 2.1.1 role testing https://review.openstack.org/353612 | 12:15 |
drifterza | hmm odd, im getting this on zuul and my local testing: Running setup.py bdist_wheel for posix-spawn: finished with status 'error'", | 12:16 |
admin0 | then here is what i will do .. i will spin up another machine first ..to run the multi-node aio .. (hopefully i can checkout stable/newton) there instead of master and run the tests | 12:16 |
admin0 | if it works there, but not on mine, i will have at least 2 systems to confirm my findings | 12:16 |
openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_cinder: Update tox.ini tests target for stable/newton https://review.openstack.org/380614 | 12:17 |
*** jrollen has quit IRC | 12:18 | |
odyssey4me | admin0 that's fine - but do the bug report with the details of your setup, then if you dig into it and figure out more information you can always add it to the bug report | 12:18 |
*** hughmFLEXin has quit IRC | 12:19 | |
admin0 | that is also correct .. will do :) | 12:20 |
openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_cinder: Update ansible-role-requirements to stable/newton https://review.openstack.org/380239 | 12:21 |
*** schwicht has quit IRC | 12:26 | |
odyssey4me | andymccr hmm, now that's a problem: http://logs.openstack.org/45/381045/3/gate/gate-openstack-ansible-os_nova-ansible-func_lxd-ubuntu-xenial/a264118/logs/host/nova/nova-compute.log.txt.gz | 12:27 |
odyssey4me | that's why https://review.openstack.org/381045 & https://review.openstack.org/381048 are failing for tempest/os_nova | 12:28 |
andymccr | it looks like another race condition :( | 12:28 |
odyssey4me | and that may be due to https://review.openstack.org/381003 :/ | 12:28 |
openstackgerrit | Merged openstack/openstack-ansible: Fix a few grammatical errors https://review.openstack.org/380802 | 12:29 |
odyssey4me | perhaps do a separate patch which combines both, then if it works just abandon those? | 12:29 |
odyssey4me | ref those in the separate patch | 12:29 |
odyssey4me | oh hang on - those are targeting different branches | 12:30 |
andymccr | it is giving the same error | 12:31 |
andymccr | hmm | 12:31 |
mgariepy | odyssey4me, can you +w this patch ? https://review.openstack.org/#/c/380776/ | 12:31 |
odyssey4me | this needs to go https://github.com/openstack/openstack-ansible-os_nova/blob/master/tasks/nova_install.yml#L17 | 12:31 |
odyssey4me | thanks to https://github.com/openstack/openstack-ansible-os_nova/blob/master/tasks/nova_install_apt.yml#L96 | 12:31 |
odyssey4me | mgariepy no, I made it - it needs two other cores to agree that it's a good patch | 12:33 |
mgariepy | ha ok ;) | 12:33 |
odyssey4me | andymccr could perhaps give it a look over | 12:33 |
andymccr | lgtm | 12:33 |
openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_nova: Remove dynamic inclusion from inventory-based service setup https://review.openstack.org/381003 | 12:36 |
mhayden | happy monday | 12:38 |
openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_nova: Remove dynamic inclusion from inventory-based service setup https://review.openstack.org/381088 | 12:38 |
openstackgerrit | Marc Gariépy proposed openstack/openstack-ansible-os_nova: Add centos support for os_nova https://review.openstack.org/378963 | 12:39 |
odyssey4me | andymccr https://review.openstack.org/381088 should solve that for master, whil I updated https://review.openstack.org/381003 to include the same for newton | 12:39 |
*** fguillot has joined #openstack-ansible | 12:40 | |
odyssey4me | mhayden andymccr evrardjp reviews of https://review.openstack.org/380999 would be appreciated | 12:40 |
evrardjp | k | 12:40 |
evrardjp | what I like with ansible, is its dual theory | 12:41 |
evrardjp | either you're fast | 12:41 |
evrardjp | either you're reliable | 12:41 |
evrardjp | it's been like that for ages now | 12:41 |
openstackgerrit | Merged openstack/openstack-ansible-os_neutron: Revert dynamic include for inventory-based conditionals https://review.openstack.org/380759 | 12:41 |
*** jrollen has joined #openstack-ansible | 12:41 | |
*** karimb has joined #openstack-ansible | 12:44 | |
*** jrollen has quit IRC | 12:45 | |
*** jrollen has joined #openstack-ansible | 12:45 | |
*** jroll has quit IRC | 12:53 | |
*** schwicht has joined #openstack-ansible | 12:54 | |
odyssey4me | andymccr mhayden need this one voted through to get stable/newton on the right path: https://review.openstack.org/380999 | 12:55 |
*** jrollen is now known as jroll | 12:55 | |
openstackgerrit | Merged openstack/openstack-ansible-tests: Provide OS-specific kernel module path https://review.openstack.org/380776 | 13:04 |
openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_neutron: Add CentOS support for os_neutron https://review.openstack.org/380491 | 13:06 |
*** klamath has joined #openstack-ansible | 13:08 | |
*** klamath has quit IRC | 13:08 | |
*** klamath has joined #openstack-ansible | 13:09 | |
openstackgerrit | Merged openstack/openstack-ansible-ops: Fix typo: remove redundant 'the' https://review.openstack.org/380921 | 13:09 |
openstackgerrit | Andy McCrae proposed openstack/openstack-ansible-tests: Add sahara vars and playbook https://review.openstack.org/381116 | 13:11 |
cloudnull | mornings | 13:11 |
admin0 | “ERROR cinder.volume.manager FunctionNotSupported: error creating image” — could be be the rbd driver ? | 13:13 |
admin0 | mornings | 13:13 |
openstackgerrit | Andy McCrae proposed openstack/openstack-ansible-os_sahara: Ansible 2.1.1 role testing https://review.openstack.org/353612 | 13:13 |
andymccr | that ansible-tests PR should make the 2.1.1 pr for sahara work now. | 13:14 |
*** goldenfri has joined #openstack-ansible | 13:14 | |
cloudnull | admin0: could be ? | 13:15 |
cloudnull | once you get away from the reference it's more likely that things will run into that. which could simply be something they don't intend to implement. | 13:16 |
admin0 | cloudnull: this is the config I used https://gist.github.com/a1git/f55bde9e46f58fb608e6bb87d9c7224c [ubuntu 16.04, stable/newton ] and this is the result I get when creating a cinder volume: https://gist.githubusercontent.com/a1git/6f24e6ffe1863241fddf98a641d599be/raw/7a8b6f61280c10604473890ff27cc4fb1eadec02/gistfile1.txt — volumeID: b84ad51e-f98f-4784-aaa8-ff7a16287b3a | 13:16 |
cloudnull | what version is the deployment? | 13:16 |
admin0 | i did a checkout of stable/newton | 13:17 |
admin0 | to test | 13:17 |
*** jperry has joined #openstack-ansible | 13:17 | |
cloudnull | is that glance backed by ceph as well? | 13:18 |
admin0 | yes | 13:18 |
admin0 | i already checked the keyrings .. all check out fine | 13:18 |
*** jperry has quit IRC | 13:18 | |
cloudnull | we're you uploading a new image or creating a snap of an existing one? | 13:19 |
admin0 | brand new install | 13:19 |
openstackgerrit | Merged openstack/openstack-ansible-os_cinder: Remove dynamic inclusion from inventory-based service setup https://review.openstack.org/380999 | 13:19 |
admin0 | trying to create a new image ; trying to create a new volume | 13:19 |
cloudnull | maybe its the client versions of ceph causing it to not talk to the backend? | 13:20 |
cloudnull | something like a mismatch as described here: https://bugs.launchpad.net/charms/+source/cinder/+bug/1581572 | 13:21 |
openstack | Launchpad bug 1581572 in cinder (Juju Charms Collection) "Upgrade L->M: FunctionNotSupported on volume-create" [Undecided,Won't fix] | 13:21 |
admin0 | ceph servers are still trusty .. everything else mitaka .. ceph-deploy fails on xenial due to systemd acting up | 13:21 |
admin0 | everything else xenial* | 13:21 |
cloudnull | mattt: andymccr: odyssey4me: you guys have any insight into this ? | 13:22 |
* cloudnull is no ceph expert | 13:22 | |
admin0 | hmm.. but ceph can also be on centos right ? | 13:22 |
odyssey4me | admin0 the ceph cluster can be on anything - assuming that ceph-auth can fetch keys between platforms | 13:23 |
cloudnull | it can | 13:23 |
admin0 | it is fetching it .. i verified that the keys are there | 13:23 |
odyssey4me | IIRC the fetch involves an ssh connection, so maybe you need keys on the cluster? | 13:23 |
odyssey4me | ok - that should be good then | 13:23 |
odyssey4me | otherwise it's been too long - cloudnull it'd be great if we could path the multi-node test to be able to do ceph so we can test it out properly | 13:24 |
automagically | Morning all | 13:24 |
mattt | what is /build/ceph-XmVvyr/ceph-10.2.2/src/build/rbd.c ? | 13:25 |
admin0 | i do ceph auth list on the ceph server and note down say cinder or glance .. then login to their respective containers and say check the /etc/ceph/ceph.client.cinder.keyring and the key is there .. same for glance | 13:25 |
admin0 | mattt: was that a question to me — coz i am not sure | 13:26 |
mattt | admin0: well yes, it's your ceph cluster no? :) | 13:26 |
admin0 | mattt: https://www.openstackfaq.com/ceph-setup-ubuntu-three-node/ — that are all the command entered to build it up | 13:27 |
admin0 | just standard ceph-deploy few lines | 13:27 |
admin0 | that /build/ stuff is from the cinder_volume_container | 13:29 |
openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible: Fix a few grammatical errors https://review.openstack.org/381122 | 13:29 |
admin0 | is_metal is false, so that container is running in one of the container nodes | 13:30 |
admin0 | controller* | 13:30 |
cloudnull | odyssey4me: ++ I know that infra has been working on making multi-node tests in the gate work better. | 13:30 |
cloudnull | so maybe in this cycle we can codify that | 13:30 |
odyssey4me | cloudnull it's not useful to us unless we can get 5 nodes or more - just two nodes adds no value to our deployment | 13:31 |
cloudnull | it may make the deployment faster by taking load down on the single host? | 13:32 |
cloudnull | mornings automagically | 13:32 |
*** schwicht has quit IRC | 13:33 | |
odyssey4me | interesting that there are no nova logs here: http://logs.openstack.org/03/381003/2/check/gate-openstack-ansible-os_nova-ansible-func-ubuntu-xenial/2197dc7/logs/host/ | 13:34 |
odyssey4me | even though nova-compute clearly was installed and restarted: http://logs.openstack.org/03/381003/2/check/gate-openstack-ansible-os_nova-ansible-func-ubuntu-xenial/2197dc7/console.html#_2016-10-03_13_10_38_446899 | 13:34 |
*** janki has quit IRC | 13:36 | |
odyssey4me | going afk for the afternoon - be back tomorrow | 13:36 |
automagically | cloudnull: Been looking at how we do the repo build as I’m wondering if it would be possible to run a mitaka build with all of the telemetry components using the newton versions. | 13:38 |
automagically | Thoughts on that? | 13:38 |
openstackgerrit | Marc Gariépy proposed openstack/openstack-ansible-os_nova: Add CentOS support for os_nova https://review.openstack.org/378963 | 13:38 |
admin0 | thoughts on what I can do now ?how to test further ? | 13:39 |
*** jperry has joined #openstack-ansible | 13:43 | |
*** bjolo has joined #openstack-ansible | 13:46 | |
*** schwicht has joined #openstack-ansible | 13:46 | |
*** fops has quit IRC | 13:51 | |
*** fops has joined #openstack-ansible | 13:52 | |
openstackgerrit | Travis Truman (automagically) proposed openstack/openstack-ansible-os_ceilometer: Cleanup test-vars handling https://review.openstack.org/381133 | 13:53 |
*** evilt0ne has joined #openstack-ansible | 13:59 | |
*** evilt0ne has quit IRC | 13:59 | |
*** evilt0ne has joined #openstack-ansible | 13:59 | |
cloudnull | automagically: sorry was cafinating | 14:00 |
automagically | np | 14:00 |
cloudnull | automagically: I think it should be possible | 14:00 |
automagically | Seems like our requirements templating would have to change pretty dramatically, no? | 14:00 |
cloudnull | they should be compatible. | 14:01 |
cloudnull | I'm assuming you want to use the repo-build role from newton and build the a mitaka env. | 14:02 |
* cloudnull may not be properly cafinated at this point though | 14:02 | |
automagically | cloudnull: Let me think through it a bit more. The primary issue would seem to be that we now define a single upper-constraints file for all wheel builds, but in this scenario, we’d effectively need multiple upper-constraints | 14:03 |
openstackgerrit | OpenStack Proposal Bot proposed openstack/openstack-ansible-ops: Updated from global requirements https://review.openstack.org/381139 | 14:03 |
openstackgerrit | OpenStack Proposal Bot proposed openstack/openstack-ansible-tests: Updated from global requirements https://review.openstack.org/381140 | 14:04 |
cloudnull | doesn't the constraint file come from openstack requirements ? | 14:05 |
cloudnull | oh, are we talking about using newton ceilo w/ a mitaka deploy? | 14:06 |
* cloudnull may not be fully awake yet... | 14:07 | |
*** Mudpuppy has joined #openstack-ansible | 14:07 | |
openstackgerrit | Merged openstack/openstack-ansible-os_tempest: Move "--no-binary libvirt-python" to ansible-tests https://review.openstack.org/381048 | 14:07 |
*** shausy has quit IRC | 14:10 | |
automagically | cloudnull: Yep, newton ceilo, aodh and gnocchi with a mitaka deploy | 14:11 |
*** jmckind has joined #openstack-ansible | 14:12 | |
*** michaelgugino has joined #openstack-ansible | 14:14 | |
cloudnull | so it "should" go. but its likely to change some of the deps used by other services within a release so yes, the multi-constraint things will likely be needed. | 14:14 |
openstackgerrit | Major Hayden proposed openstack/openstack-ansible-security: Update tox.ini tests target for stable/newton https://review.openstack.org/380635 | 14:17 |
*** spotz_zzz is now known as spotz | 14:17 | |
*** woodard has joined #openstack-ansible | 14:18 | |
automagically | cloudnull: Thanks for letting me bounce that off of you. I’ll play around with it a bit and see where I get | 14:18 |
*** galstrom_zzz is now known as galstrom | 14:19 | |
cloudnull | In pondering that, I think it may be best to build it in an isolated env only selecting to build ceilo (and its components) then sync the results to your main set of repo servers. | 14:20 |
cloudnull | similar to how we do the multi-arch stuff | 14:20 |
openstackgerrit | Marc Gariépy proposed openstack/openstack-ansible-tests: Fix CentOS package installation https://review.openstack.org/381159 | 14:23 |
*** chris_hultin|AWA is now known as chris_hultin | 14:24 | |
automagically | One last question for you cloudnull - any idea WTH is going on here: http://logs.openstack.org/33/381133/1/check/gate-openstack-ansible-os_ceilometer-ansible-func_gnocchi-ubuntu-xenial-nv/23dc617/console.html#_2016-10-03_13_54_54_410113 | 14:27 |
* cloudnull looking | 14:27 | |
*** michaelgugino has quit IRC | 14:28 | |
* kysse_ :C | 14:29 | |
*** kysse_ is now known as kysse | 14:29 | |
*** michaelgugino has joined #openstack-ansible | 14:30 | |
vnogin | guys, does anyone know in which playbook we execute dynamic_inventory.py script? :) | 14:33 |
*** DanyC has joined #openstack-ansible | 14:33 | |
automagically | vnogin: Ansible itself executes it to generate the inventory | 14:33 |
cloudnull | automagically: i have no idea. | 14:34 |
automagically | Appreciate you looking, maybe I’ll bug infra | 14:34 |
cloudnull | it looks like something within pip. | 14:34 |
*** woodard has quit IRC | 14:34 | |
cloudnull | but i have no clue | 14:34 |
vnogin | automagically, tnx | 14:35 |
admin0 | any suggestions for me :D ? | 14:35 |
admin0 | i am kinda stuck | 14:35 |
admin0 | i redeployed twice, no help .. no errors | 14:35 |
ikalnytskyi | automagically: vnogin which mechanism is lied under that execution? according to ansible docs, the script should be passed via -i option. i didn't find any passing. :( | 14:35 |
admin0 | maybe the drives linked are different | 14:35 |
cloudnull | automagically: rather tox invoking pip via python subprocess (based on the stacktrace above) | 14:36 |
automagically | ikalnytskyi: https://github.com/openstack/openstack-ansible/blob/master/scripts/openstack-ansible.rc | 14:36 |
cloudnull | admin0: I personally am at a loss with ceph. i did notice in your paste with the logs that you were running 14.0.0 which is newton RC. maybe its something within the RC that's not working (either from openstack-ansible, or from upstream cinder) ? | 14:38 |
admin0 | yeah, wantted to help verify ceph stuff before we mark it as stable | 14:39 |
michaelgugino | is there a bug filed? | 14:40 |
michaelgugino | I've been working with ceph playbook heavily last couple of weeks | 14:40 |
admin0 | does it work for you ? | 14:40 |
ikalnytskyi | automagically: so it's env variable, thank you! did you know anything about stable/mitaka branch? there's no such file there | 14:41 |
vnogin | automagically, ikalnytskyi it's correct for newton, and what about mitaka? | 14:41 |
michaelgugino | does what work? ceph_client role? Yes | 14:41 |
*** chhavi has quit IRC | 14:41 | |
*** TxGirlGeek has joined #openstack-ansible | 14:42 | |
admin0 | michaelgugino: , this is what i used to setup ceph: https://www.openstackfaq.com/ceph-setup-ubuntu-three-node/ , config I used: https://gist.github.com/a1git/f55bde9e46f58fb608e6bb87d9c7224c and the errors I get: https://gist.githubusercontent.com/a1git/6f24e6ffe1863241fddf98a641d599be/raw/7a8b6f61280c10604473890ff27cc4fb1eadec02/gistfile1.txt | 14:43 |
openstackgerrit | Major Hayden proposed openstack/openstack-ansible-security: Update tox.ini tests target for stable/newton https://review.openstack.org/380635 | 14:44 |
michaelgugino | let me see... | 14:44 |
admin0 | if you can PM me your ssh key, you can have access to there as well :) | 14:44 |
*** psilvad has quit IRC | 14:45 | |
*** Guest95 has joined #openstack-ansible | 14:45 | |
*** psilvad has joined #openstack-ansible | 14:46 | |
ikalnytskyi | vnogin: automagically it seems i got it - previously there were an ansible.cfg https://github.com/openstack/openstack-ansible/blob/stable/mitaka/playbooks/ansible.cfg thx for help. | 14:49 |
automagically | Yep, that’s it ikalnytskyi | 14:49 |
admin0 | michaelgugino: https://gist.github.com/a1git/50707330835c430c2fff11b5209e9968 — latest I tried .. logs | 14:50 |
openstackgerrit | Marc Gariépy proposed openstack/openstack-ansible-os_neutron: Add CentOS support for os_neutron https://review.openstack.org/380491 | 14:52 |
michaelgugino | https://bugs.launchpad.net/charms/+source/cinder/+bug/1581572 | 14:52 |
openstack | Launchpad bug 1581572 in cinder (Juju Charms Collection) "Upgrade L->M: FunctionNotSupported on volume-create" [Undecided,Won't fix] | 14:52 |
michaelgugino | possibly a mismatch between ceph client version and ceph backend | 14:53 |
mgariepy | can someone review this please ? https://review.openstack.org/#/c/381159/ | 14:53 |
michaelgugino | admin0: ^ | 14:53 |
automagically | mgariepy: Looking | 14:54 |
*** DanyC has quit IRC | 14:55 | |
admin0 | michaelgugino this means people need to upgrade ceph before upgrading to newton .. .. shouldn’t ansible check the verison and install appropriate libraries ? | 14:55 |
michaelgugino | I'm not sure. What version of ceph do you have deployed? | 14:56 |
*** phalmos has joined #openstack-ansible | 14:57 | |
admin0 | default that comes with ubuntu 14.04 ( 16.04 failed due to some systemd issue) | 14:57 |
admin0 | hecking | 14:57 |
admin0 | checking! | 14:57 |
admin0 | ceph version 0.80.11 | 14:57 |
*** vishwanathj has joined #openstack-ansible | 14:58 | |
michaelgugino | admin0: that's a really old version of ceph. I believe ceph is usually installed directly from upstream repos. I suggest looking into ceph-ansible, that's what we use | 14:59 |
*** galstrom is now known as galstrom_zzz | 15:01 | |
*** kstev has joined #openstack-ansible | 15:03 | |
admin0 | i will re-setup the ceph with the latest to check out other featuers .. but thats a big assumption to base the client version on .. my thought was it will ssh, determine the correct version and install the appropriate versions | 15:03 |
admin0 | due to n-1 or release-1 or other *company* requirements, people also use the default that comes in the OS itself | 15:04 |
*** fguillot has quit IRC | 15:04 | |
mattt | admin0: you can use distro | 15:10 |
openstackgerrit | Nolan Brubaker proposed openstack/openstack-ansible: Return append status, use it for accurate logging https://review.openstack.org/380640 | 15:10 |
mattt | admin0: https://github.com/openstack/openstack-ansible-ceph_client/blob/master/defaults/main.yml#L32-L45 | 15:10 |
mattt | perhaps you need to adjust your config | 15:10 |
admin0 | trigger happy me :) ceph servers are already rebuilding :D | 15:11 |
admin0 | but will try that first :) | 15:11 |
*** kylek3h has joined #openstack-ansible | 15:12 | |
*** Fenuks has joined #openstack-ansible | 15:13 | |
admin0 | mattt: my controllers and compute is in ubuntu 16.04 .. my ceph is in 14.04 (but can easily be centos as well) | 15:13 |
admin0 | in these situations, how does this variable work ? | 15:13 |
Fenuks | Hello. Can anyone help? Getting error "Could not find a version that satisfies the requirement ndg-httpsclient (from versions: ) | 15:13 |
admin0 | if I set it to distro, is it the distro of the ceph mons or distro of the controllers ? | 15:13 |
Fenuks | " when trying to run setup-hosts playbook | 15:14 |
Fenuks | OSA version 13.3.4 | 15:15 |
admin0 | because in large deployments ( like 2-3 avaibality zones) you want to do rolling upgrades | 15:15 |
admin0 | this would mean upgrade everything at once .. ceph + controlles or ceph + controllers + compute — so some confusion on the version part | 15:16 |
*** pcaruana has quit IRC | 15:18 | |
admin0 | ceph_pkg_source: ceph = same as ceph-ansible i believe .. which is why it works everytime ! | 15:18 |
mattt | admin0: those configs control which client packages are installed on your openstack-nodes, so your cinder-volume, glance-api, nova-compute, etc. | 15:18 |
admin0 | ok | 15:19 |
openstackgerrit | Merged openstack/openstack-ansible: Fix a few grammatical errors https://review.openstack.org/381122 | 15:22 |
admin0 | going-home time .. will use the ceph-ansible way and give it a try :) | 15:23 |
admin0 | thanks for the pointers :) | 15:24 |
*** admin0 has quit IRC | 15:25 | |
*** admin0 has joined #openstack-ansible | 15:25 | |
*** admin0 has quit IRC | 15:25 | |
michaelgugino | odyssey4me: your fix didn't work for my cinder patch | 15:28 |
Fenuks | Anyone? pip search finds the package, there's a page on pypi, it just refuses to download and install it | 15:28 |
michaelgugino | Fenuks, what step are you on? What play is failing? | 15:29 |
michaelgugino | are you behind a web proxy? | 15:29 |
openstackgerrit | Merged openstack/openstack-ansible-tests: Updated from global requirements https://review.openstack.org/381140 | 15:30 |
Fenuks | michaelgugino: No proxy. Playb is lxc-hosts-setup.yml from setup-hosts.yml | 15:30 |
openstackgerrit | Merged openstack/openstack-ansible-tests: Fix CentOS package installation https://review.openstack.org/381159 | 15:31 |
Fenuks | Error is "failed: [nir] => {"attempts": 5, "cmd": "/usr/local/bin/pip install -U ndg-httpsclient requests", "failed": true}". Package requiests is installed and up to date. | 15:32 |
*** cathrichardson has joined #openstack-ansible | 15:32 | |
openstackgerrit | Major Hayden proposed openstack/openstack-ansible-security: [TEST] Shaking out gate issues https://review.openstack.org/380621 | 15:32 |
automagically | odyssey4me and andymccr - Thoughts on how best to resolve this issue: https://bugs.launchpad.net/openstack-ansible/+bug/1628952/comments/3 | 15:34 |
openstack | Launchpad bug 1628952 in openstack-ansible "os_ceilometer role missing functional gate testing of gnocchi storage option" [Undecided,Fix released] - Assigned to Travis Truman (travis-truman) | 15:34 |
automagically | See https://bugs.launchpad.net/openstack-ansible/+bug/1628952/comments/4 for my amateur attempt at a solution that was show down by infra | 15:34 |
*** AnarchyAo has quit IRC | 15:35 | |
michaelgugino | Fenuks: I'm really not sure. Sometimes there are problems if you install pip with apt-get install python-pip. There have been some changes with the openstack requirements constraints files recently. You'll have to do some debugging on your machine locally to figure out why pip is failing. | 15:35 |
Fenuks | michaelgugino: It looks for package in repo-container, but I'm not sure where repo-containter looks for them and how it's populated… | 15:37 |
michaelgugino | automagically: what is the test name just too long? | 15:37 |
michaelgugino | Fenuks: if you're just running setup-hosts, I don't think the repo container is built yet | 15:37 |
automagically | michaelgugino: Looks like tox pukes somehow because of the length of the directory name. Length of the directory name is directly influenced by the jenkins job name | 15:37 |
Fenuks | michaelgugino: I'm upgrading. For some reason, perhaps becouse swift is installed baremetal, node is trying to use repo container | 15:38 |
andymccr | automagically: could change the name to "gnocchi" | 15:39 |
andymccr | other than that i cant think of a better way - similarly in swift we named the tests like "multi_reg" instead of func_multi_reg because of the length. | 15:39 |
openstackgerrit | Travis Truman (automagically) proposed openstack/openstack-ansible-os_ceilometer: Cleanup test-vars handling https://review.openstack.org/381133 | 15:39 |
automagically | andymccr: Fair point, will give that a swing | 15:39 |
palendae | andymccr, Did you get the ring script testing started that you had asked about at midcycle? | 15:42 |
andymccr | palendae: no :( its on my todo list - i had a skeleton patch in, with the (good) intention of working that, but I'm trying to get virtualbmc to work for ironic now as a greater prio | 15:43 |
evrardjp | I'm off for today | 15:43 |
*** mfisch` is now known as mfisch | 15:43 | |
palendae | andymccr, np, I may take a look between this inventory bug cause it's hurting my head | 15:43 |
andymccr | palendae: that would be awesome. | 15:43 |
*** mfisch is now known as Guest7223 | 15:43 | |
openstackgerrit | Major Hayden proposed openstack/openstack-ansible-security: [TEST] Shaking out gate issues https://review.openstack.org/380621 | 15:43 |
* palendae goes to find the patch in gerrit | 15:45 | |
openstackgerrit | Travis Truman (automagically) proposed openstack/openstack-ansible-os_ceilometer: Shorten the gnocchi scenario name https://review.openstack.org/381200 | 15:46 |
drifterza | anyone else getting this error: Running setup.py bdist_wheel for posix-spawn: finished with status 'error'" | 15:48 |
*** galstrom_zzz is now known as galstrom | 15:48 | |
drifterza | I get it on zuul and my local aio | 15:48 |
openstackgerrit | Donovan Francesco proposed openstack/openstack-ansible: Add integrated Trove deployment. https://review.openstack.org/374008 | 15:48 |
*** Guest7223 is now known as mfisch | 15:49 | |
*** mfisch has quit IRC | 15:49 | |
*** mfisch has joined #openstack-ansible | 15:49 | |
mgariepy | odyssey4me, is rebase necesary when depending on another patch ? | 15:51 |
*** bgmccollum has joined #openstack-ansible | 15:53 | |
openstackgerrit | Major Hayden proposed openstack/openstack-ansible-security: [TEST] Shaking out gate issues https://review.openstack.org/380621 | 15:55 |
*** chhavi has joined #openstack-ansible | 15:56 | |
mhayden | wow, something is causing chrony to stop in the gate | 15:57 |
palendae | andymccr, Was your skeleton patch on gerrit? Not seeing it https://review.openstack.org/#/q/owner:%22Andy+McCrae+%253Candy.mccrae%2540googlemail.com%253E%22 | 15:58 |
andymccr | palendae: lemme look | 15:58 |
andymccr | ahh i got a lot less far than i thought... https://review.openstack.org/#/c/353075/ i basically got it to clone out the script so i could test it locally as part of a run. sorry, i thought i had already done some of the test startup | 15:59 |
palendae | np | 16:00 |
palendae | Out for lunch | 16:00 |
*** fguillot has joined #openstack-ansible | 16:00 | |
openstackgerrit | Major Hayden proposed openstack/openstack-ansible-security: [TEST] Shaking out gate issues https://review.openstack.org/380621 | 16:03 |
*** dragonmaster has quit IRC | 16:03 | |
*** asettle__ has joined #openstack-ansible | 16:10 | |
openstackgerrit | Deepak proposed openstack/openstack-ansible-tests: Renamed test-vars.yml to rolename-overrides.yml in README.rst https://review.openstack.org/381212 | 16:10 |
openstackgerrit | Donovan Francesco proposed openstack/openstack-ansible: Adding Trove Horizon dashboard https://review.openstack.org/375251 | 16:11 |
*** asettle has quit IRC | 16:14 | |
*** asettle__ has quit IRC | 16:15 | |
automagically | andymccr: Can I get your reviews on https://review.openstack.org/#/c/381185/ and https://review.openstack.org/#/c/381200/ please | 16:17 |
mhayden | is it possible to see the /var/log/syslog in a gate job? i keep getting Permission Denied, even with sudo | 16:20 |
cloudnull | so this is happening in ansible 2.2 http://docs.ansible.com/ansible/include_role_module.html | 16:20 |
cloudnull | which may feed well into some great improvements. | 16:21 |
mhayden | oh that's fancy | 16:21 |
*** hughmFLEXin has joined #openstack-ansible | 16:23 | |
openstackgerrit | Major Hayden proposed openstack/openstack-ansible-security: [TEST] Shaking out gate issues https://review.openstack.org/380621 | 16:24 |
cloudnull | where are we with newton | 16:29 |
cloudnull | do we have anything that needs to be specifically hammered on | 16:29 |
cloudnull | ? | 16:29 |
*** sdake has quit IRC | 16:30 | |
*** hughmFLEXin has quit IRC | 16:31 | |
openstackgerrit | Merged openstack/openstack-ansible-tests: Add sahara vars and playbook https://review.openstack.org/381116 | 16:31 |
*** hughmFLEXin has joined #openstack-ansible | 16:32 | |
*** karimb has quit IRC | 16:33 | |
*** sdake has joined #openstack-ansible | 16:34 | |
*** drifterza has quit IRC | 16:35 | |
cloudnull | andymccr odyssey4me -- bump | 16:36 |
openstackgerrit | Major Hayden proposed openstack/openstack-ansible-security: Skip V-38620 (chrony) in gate https://review.openstack.org/381236 | 16:36 |
openstackgerrit | Major Hayden proposed openstack/openstack-ansible-security: Initial scaffolding for RHEL 7 STIG https://review.openstack.org/380509 | 16:37 |
cloudnull | woot! | 16:37 |
cloudnull | moar STIG | 16:38 |
mhayden | yarly | 16:38 |
mhayden | ntpd is running in the gate jobs and chrony can't bind :| | 16:38 |
mhayden | alas, it must be skippered | 16:38 |
*** DanyC has joined #openstack-ansible | 16:44 | |
*** admin0 has joined #openstack-ansible | 16:44 | |
openstackgerrit | Travis Truman (automagically) proposed openstack/openstack-ansible-os_heat: add os_heat role functional tests for gate https://review.openstack.org/378024 | 16:47 |
*** deepak_jon has joined #openstack-ansible | 16:47 | |
*** DanyC has quit IRC | 16:48 | |
openstackgerrit | Merged openstack/openstack-ansible-plugins: Adding Magnum plugin https://review.openstack.org/378029 | 16:54 |
*** chhavi has quit IRC | 16:54 | |
mhayden | cloudnull: could you give me a look on that ntpd shenanigans? https://review.openstack.org/381236 | 16:55 |
cloudnull | mhayden: sure thing | 16:58 |
mhayden | cloudnull: thanks mucho | 16:58 |
openstackgerrit | Merged openstack/openstack-ansible-security: Skip V-38620 (chrony) in gate https://review.openstack.org/381236 | 17:06 |
openstackgerrit | Major Hayden proposed openstack/openstack-ansible-security: Skip V-38620 (chrony) in gate https://review.openstack.org/381251 | 17:09 |
admin0 | in our setup, what is the monitor_interface ? is it br-storage ? | 17:12 |
*** AnarchyAo has joined #openstack-ansible | 17:12 | |
openstackgerrit | Deepak proposed openstack/openstack-ansible-tests: Renamed test-vars.yml to rolename-overrides.yml in README.rst https://review.openstack.org/381212 | 17:13 |
openstackgerrit | Major Hayden proposed openstack/openstack-ansible-security: Skip V-38620 (chrony) in gate https://review.openstack.org/381254 | 17:30 |
openstackgerrit | Major Hayden proposed openstack/openstack-ansible-security: Skip V-38620 (chrony) in gate https://review.openstack.org/381255 | 17:30 |
*** hughmFLEXin has quit IRC | 17:37 | |
openstackgerrit | Major Hayden proposed openstack/openstack-ansible-security: Update tox.ini tests target for stable/newton https://review.openstack.org/380635 | 17:37 |
*** hughmFLEXin has joined #openstack-ansible | 17:37 | |
*** sdake has quit IRC | 17:39 | |
openstackgerrit | Deepak proposed openstack/openstack-ansible-os_ceilometer: Update test-vars.yml for repositories https://review.openstack.org/381257 | 17:39 |
*** electrofelix has quit IRC | 17:40 | |
*** sdake has joined #openstack-ansible | 17:44 | |
*** adrian_otto has joined #openstack-ansible | 17:49 | |
*** sdake has quit IRC | 18:01 | |
*** poopcat has joined #openstack-ansible | 18:01 | |
*** drifterza has joined #openstack-ansible | 18:01 | |
*** poopcat has quit IRC | 18:10 | |
*** poopcat has joined #openstack-ansible | 18:10 | |
openstackgerrit | Deepak proposed openstack/openstack-ansible-os_cinder: Update test-vars.yml for repositories https://review.openstack.org/381267 | 18:10 |
openstackgerrit | Michael Gugino proposed openstack/openstack-ansible-os_cinder: Implement support for cinder qos specs https://review.openstack.org/375131 | 18:11 |
*** asettle has joined #openstack-ansible | 18:11 | |
*** Fenuks has quit IRC | 18:17 | |
*** hughmFLEXin has quit IRC | 18:17 | |
*** jorge_munoz_ has joined #openstack-ansible | 18:17 | |
*** jorge_munoz has quit IRC | 18:18 | |
*** jorge_munoz_ is now known as jorge_munoz | 18:18 | |
*** hughmFLEXin has joined #openstack-ansible | 18:19 | |
*** mkrish004 has joined #openstack-ansible | 18:19 | |
*** woodard has joined #openstack-ansible | 18:19 | |
*** weezS has joined #openstack-ansible | 18:20 | |
openstackgerrit | Jimmy McCrory proposed openstack/openstack-ansible-os_magnum: Add basic functional tests to role https://review.openstack.org/355155 | 18:24 |
openstackgerrit | Merged openstack/openstack-ansible-tests: Renamed test-vars.yml to rolename-overrides.yml in README.rst https://review.openstack.org/381212 | 18:28 |
*** pcaruana has joined #openstack-ansible | 18:29 | |
openstackgerrit | Travis Truman (automagically) proposed openstack/openstack-ansible-os_cinder: Remove duplicate delegate_to https://review.openstack.org/381269 | 18:30 |
*** woodard has quit IRC | 18:32 | |
*** fops has quit IRC | 18:32 | |
*** hughmFLEXin has quit IRC | 18:35 | |
*** hughmFLEXin has joined #openstack-ansible | 18:36 | |
andymccr | https://review.openstack.org/#/c/353612/ - sahara 2.1.1 ready to go! | 18:36 |
mattt | i need to catch up on all these testing changes :( | 18:45 |
andymccr | mattt: that one is a bit of a 2 parter, since its moving to ansible 2.1.1 as well as central testing repo | 18:46 |
*** admin0 has quit IRC | 18:47 | |
andymccr | but we have a readme in the openstack-ansible-tests repo that might help, if not let me know and we'll try improverate it. | 18:47 |
mattt | andymccr: reading now | 18:47 |
openstackgerrit | Merged openstack/openstack-ansible-os_ceilometer: Cleanup test-vars handling https://review.openstack.org/381133 | 18:49 |
*** drifterza has quit IRC | 18:50 | |
cloudnull | andymccr: well done | 18:52 |
*** asettle has quit IRC | 19:03 | |
*** Guest95 has quit IRC | 19:04 | |
*** cbits has quit IRC | 19:06 | |
*** hughmFLEXin has quit IRC | 19:07 | |
*** hughmFLEXin has joined #openstack-ansible | 19:07 | |
automagically | cloudnull or mhayden - Could I get a quick review on https://review.openstack.org/#/c/381200/ | 19:09 |
*** phalmos has quit IRC | 19:10 | |
*** AnarchyAo has quit IRC | 19:10 | |
*** alij has joined #openstack-ansible | 19:10 | |
*** TxGirlGeek has quit IRC | 19:10 | |
alij | Hi, can anyone tell me what can be the reason that at times, suddenly, ansible is unable to SSH into some containers? | 19:11 |
openstackgerrit | Marc Gariépy proposed openstack/openstack-ansible-galera_server: Cleanup duplicate key warning. https://review.openstack.org/381273 | 19:11 |
openstackgerrit | Merged openstack/openstack-ansible-os_sahara: Ansible 2.1.1 role testing https://review.openstack.org/353612 | 19:11 |
*** asettle has joined #openstack-ansible | 19:12 | |
alij | Even though the cluster was brought up successfully and working fine for a couple of weeks. | 19:12 |
*** hughmFLEXin has quit IRC | 19:13 | |
mhayden | automagically: gandering | 19:14 |
mhayden | automagically: aww, cloudnull sniped | 19:15 |
automagically | mhayden: cloudnull pushed it across the finish line | 19:15 |
*** hughmFLEXin has joined #openstack-ansible | 19:15 | |
mhayden | alij: are you able to manually ssh to those containers from your deployment host? | 19:15 |
alij | mhayden, what user name is used to ssh manually? | 19:18 |
alij | currently repo container is not being SSH'd by ansible | 19:19 |
alij | fatal: [infra1_repo_container-d2be75af] => SSH Error: data could not be sent to the remote host. Make sure this host can be reached over ssh | 19:19 |
*** hughmFLEXin has quit IRC | 19:20 | |
*** hughmFLEXin has joined #openstack-ansible | 19:20 | |
*** johnmilton has quit IRC | 19:21 | |
*** catintheroof has joined #openstack-ansible | 19:22 | |
*** sdake has joined #openstack-ansible | 19:22 | |
*** sdake has quit IRC | 19:22 | |
mhayden | alij: you should be able to ssh as root from your deployment host to that container | 19:23 |
alij | yeah tried that, not working | 19:23 |
alij | 22: No route to host | 19:23 |
*** Mudpuppy has quit IRC | 19:26 | |
mhayden | there may be some networking issues in your deployment | 19:26 |
mhayden | is this a multi-node deployment or a single server (AIO)? | 19:27 |
openstackgerrit | Jimmy McCrory proposed openstack/openstack-ansible-os_magnum: Add basic functional tests to role https://review.openstack.org/355155 | 19:27 |
jmccrory | chris_hultin: it's passing now | 19:28 |
chris_hultin | jmccrory: Awesome | 19:28 |
openstackgerrit | Travis Truman (automagically) proposed openstack/openstack-ansible-os_heat: Add os_heat role functional tests for gate https://review.openstack.org/378024 | 19:28 |
alij | mhayden, its a multinode setup | 19:28 |
jmccrory | had to add haproxy to route through from vm to containers | 19:28 |
alij | Okay, I see the problem. The actual IP of the container and the one in openstack_inventory.json is different :O | 19:28 |
alij | Do containers change their IP over time? | 19:29 |
alij | mhayden: | 19:31 |
mhayden | alij: they shouldn't ;) | 19:31 |
alij | mhayden, I removed the openstack_inventory.json file (so a new one is created). Same thing is happening with another container | 19:32 |
*** fops has joined #openstack-ansible | 19:32 | |
alij | Its IP does not match the actual one | 19:32 |
openstackgerrit | Merged openstack/openstack-ansible-os_ceilometer: Shorten the gnocchi scenario name https://review.openstack.org/381200 | 19:33 |
*** deepak_jon has quit IRC | 19:35 | |
*** johnmilton has joined #openstack-ansible | 19:35 | |
*** spotz is now known as spotz_zzz | 19:36 | |
*** fops has quit IRC | 19:38 | |
michaelgugino | odyssey4me: automagically cloudnull mhayden if anyone can look at my comments at https://review.openstack.org/#/c/375131/16 and help me out, that would be great | 19:39 |
mhayden | michaelgugino: whoa | 19:39 |
mhayden | that's unusual | 19:39 |
michaelgugino | indeed | 19:39 |
*** johnmilton has quit IRC | 19:39 | |
*** hughmFLEXin has quit IRC | 19:40 | |
* automagically looking | 19:40 | |
* cloudnull looking | 19:40 | |
*** johnmilton has joined #openstack-ansible | 19:41 | |
*** hughmFLEXin has joined #openstack-ansible | 19:43 | |
*** alij has quit IRC | 19:45 | |
*** TxGirlGeek has joined #openstack-ansible | 19:46 | |
*** asettle has quit IRC | 19:46 | |
*** david-lyle has joined #openstack-ansible | 19:47 | |
openstackgerrit | Jimmy McCrory proposed openstack/openstack-ansible-os_magnum: Add basic functional tests to role https://review.openstack.org/355155 | 19:48 |
*** admin0 has joined #openstack-ansible | 19:48 | |
automagically | michaelgugino: I wonder if we add an inventory entry for 127.0.01 ansible_become=True if it will go away | 19:49 |
automagically | Looks like you’ve found an Ansible bug there related to “localhost” magic | 19:49 |
michaelgugino | Why this crap decides to only affect my patch sets, I'll never know. | 19:50 |
*** bjolo has quit IRC | 19:51 | |
*** alij has joined #openstack-ansible | 19:52 | |
automagically | Hey, at least you didn’t get bit by https://github.com/ansible/ansible/issues/17755 | 19:52 |
*** alij_ has joined #openstack-ansible | 19:52 | |
*** fops has joined #openstack-ansible | 19:53 | |
*** hughmFLEXin has quit IRC | 19:54 | |
michaelgugino | wtf, alien emoji? | 19:55 |
*** alij_ has quit IRC | 19:56 | |
*** alij has quit IRC | 19:56 | |
mgariepy | lol. just because an emoji is much more clearer than plain ascii english. | 19:57 |
michaelgugino | 👽 | 20:03 |
*** sdake has joined #openstack-ansible | 20:05 | |
*** hughmFLEXin has joined #openstack-ansible | 20:06 | |
openstackgerrit | Travis Truman (automagically) proposed openstack/openstack-ansible-os_ceilometer: Add storage details to role docs https://review.openstack.org/381290 | 20:10 |
admin0 | glance + ceph worked :) | 20:10 |
admin0 | cehcking cinder + ceph | 20:10 |
openstackgerrit | Travis Truman (automagically) proposed openstack/openstack-ansible-os_ceilometer: Remove unused tenant_name var https://review.openstack.org/381291 | 20:14 |
openstackgerrit | Major Hayden proposed openstack/openstack-ansible-security: Initial docs scaffolding for RHEL 7 STIG https://review.openstack.org/381292 | 20:16 |
mhayden | michaelgugino: could it be something within /etc/hosts not recognizing localhost? | 20:16 |
mhayden | cloudnull: i backported that V-38620 skippage if you have a moment for ganderization https://review.openstack.org/381251 | 20:17 |
cloudnull | mhayden: done | 20:20 |
automagically | Hmm, upon recheck CentOS gate passed for the os_cinder role... | 20:21 |
*** hughmFLEXin has quit IRC | 20:25 | |
* mhayden hugs cloudnull | 20:25 | |
openstackgerrit | Merged openstack/openstack-ansible-security: Skip V-38620 (chrony) in gate https://review.openstack.org/381251 | 20:25 |
openstackgerrit | Merged openstack/openstack-ansible-security: Update tox.ini tests target for stable/newton https://review.openstack.org/380635 | 20:25 |
openstackgerrit | Jimmy McCrory proposed openstack/openstack-ansible-tests: Allow db user name differing from db name https://review.openstack.org/381297 | 20:28 |
*** catintheroof has quit IRC | 20:29 | |
*** david-lyle has quit IRC | 20:30 | |
*** hughmFLEXin has joined #openstack-ansible | 20:30 | |
*** drifterza has joined #openstack-ansible | 20:31 | |
drifterza | cloudnull, you around | 20:31 |
drifterza | ? | 20:31 |
drifterza | can someone review : https://review.openstack.org/#/c/375251/ , https://review.openstack.org/#/c/374008/ | 20:32 |
drifterza | please | 20:32 |
mhayden | drifterza: i'll look | 20:33 |
drifterza | thanks major | 20:34 |
cloudnull | drifterza: kinda. | 20:34 |
cloudnull | I'll have a look at those in a little while. | 20:34 |
*** david-lyle has joined #openstack-ansible | 20:35 | |
*** Jeffrey4l__ has joined #openstack-ansible | 20:35 | |
drifterza | I think they pretty solid | 20:35 |
drifterza | so shouldnt be to hectic | 20:35 |
*** Jeffrey4l_ has quit IRC | 20:35 | |
*** spotz_zzz is now known as spotz | 20:36 | |
*** phalmos has joined #openstack-ansible | 20:38 | |
openstackgerrit | ianeta hutchinson proposed openstack/openstack-ansible: [docs] Applying edits to the OSA install guide: overview https://review.openstack.org/381301 | 20:41 |
admin0 | cinder + ceph working as well :D | 20:44 |
admin0 | \o/ | 20:44 |
cloudnull | admin0: what was the change needed? | 20:44 |
cloudnull | somthing within OSA that we need to adjust? | 20:44 |
*** david-lyle has quit IRC | 20:44 | |
*** pcaruana has quit IRC | 20:45 | |
admin0 | i had this: controllers + compute = ubuntu 16.04 , ceph servers = 14.04 … ceph servers were 0.8 version, however the conttroller+compute got 0.10 version | 20:45 |
admin0 | so now i destroyed the ceph server and then used the ceph-ansible to create it, so that both point to the ceph* as the version to use | 20:45 |
drifterza | yeah ceph doesnt like major version changes | 20:47 |
drifterza | better to install using upstream repo's and perhaps version lock | 20:48 |
admin0 | yeah, but not always possible that way . .best would be to detect the version in the server and ln -s the approprirate client version :) | 20:48 |
admin0 | waiting for the ansible run to finsh, so that i finally test nova + ceph | 20:48 |
admin0 | and then can move ahead with other tests | 20:49 |
cloudnull | awesome! | 20:50 |
openstackgerrit | Jimmy McCrory proposed openstack/openstack-ansible-tests: Checksum all traffic traveling though the bridges https://review.openstack.org/381305 | 20:55 |
drifterza | cloudnull, any chance of getting a repo called openstack-ansible-os_contrail or openstack-ansible-os_opencontrail ? | 20:58 |
*** thorst has quit IRC | 20:58 | |
*** jmckind has quit IRC | 21:00 | |
cloudnull | odyssey4me: andymccr: ^ | 21:00 |
cloudnull | drifterza: I'd say yes. however odyssey4me and andymccr will need to be the ones to get it started. | 21:01 |
*** hughmFLEXin has quit IRC | 21:01 | |
cloudnull | if you have a repo that you've been working on we can import that repo at a later time | 21:01 |
cloudnull | which makes it easy to build out a POC and then import into the namespace when ready. | 21:02 |
openstackgerrit | Jimmy McCrory proposed openstack/openstack-ansible-os_magnum: Add basic functional tests to role https://review.openstack.org/355155 | 21:02 |
*** retreved has quit IRC | 21:02 | |
openstackgerrit | ianeta hutchinson proposed openstack/openstack-ansible: [DOCS] Applies edits to the OSA install guide appendix A https://review.openstack.org/381306 | 21:03 |
admin0 | what would produce an error like this ? instance: 8a8780c4-3063-4e22-82d2-5e25ba359b12] Instance was not mapped to a cell, likely due to an older nova-api service running. _populate_instance_mapping /openstack/venvs/nova-14.0.0/lib/python2.7/site-packages/nova/conductor/manager.py:436 | 21:06 |
drifterza | yeah was hoping to use zuul to get gating tests | 21:07 |
drifterza | how can i do that on my own repo | 21:07 |
admin0 | glance+ceph, cinder+ceph works fine .. nova+ceph = nope :( | 21:09 |
admin0 | will remove the nova-api containers and check after a re-do | 21:10 |
*** adrian_otto has quit IRC | 21:10 | |
cloudnull | drifterza: IDK TBH. | 21:18 |
cloudnull | i've never done that. | 21:18 |
cloudnull | personal repo > zuul > nodepool < post comment. | 21:19 |
*** openstackgerrit has quit IRC | 21:19 | |
cloudnull | hum.. | 21:19 |
cloudnull | you might be able to do that with gerrithub | 21:19 |
cloudnull | which can be an overlay on a personal repo | 21:19 |
*** david-lyle has joined #openstack-ansible | 21:19 | |
*** openstackgerrit has joined #openstack-ansible | 21:19 | |
cloudnull | we used to have stackforge for this perpose but that's been decommissioned. | 21:19 |
cloudnull | im not sure. | 21:20 |
*** schwicht has quit IRC | 21:20 | |
cloudnull | I'd recommend asking openstack-infra to see if they have some insight on if zuul+nodepool can be used outside of the OpenStack echosystem | 21:20 |
cloudnull | without having to run your own zuul that it | 21:20 |
cloudnull | *that is | 21:20 |
*** thorst has joined #openstack-ansible | 21:21 | |
openstackgerrit | Merged openstack/openstack-ansible-ops: Updated from global requirements https://review.openstack.org/381139 | 21:22 |
*** alij has joined #openstack-ansible | 21:23 | |
drifterza | eish it would be better if I can get a repo to use the same build process like the others. | 21:23 |
drifterza | is there anything outstanding for newton | 21:24 |
drifterza | ? | 21:24 |
openstackgerrit | ianeta hutchinson proposed openstack/openstack-ansible: [DOCS] Applies edits to the OSA install guide appendix B https://review.openstack.org/381312 | 21:24 |
*** thorst has quit IRC | 21:26 | |
*** alij has quit IRC | 21:28 | |
*** hughmFLEXin has joined #openstack-ansible | 21:34 | |
mrda | Morning OSA | 21:36 |
openstackgerrit | Merged openstack/openstack-ansible-galera_server: Cleanup duplicate key warning. https://review.openstack.org/381273 | 21:39 |
*** hpe-hj has joined #openstack-ansible | 21:40 | |
*** Mudpuppy has joined #openstack-ansible | 21:40 | |
*** psilvad has quit IRC | 21:42 | |
*** hj-hpe has quit IRC | 21:42 | |
drifterza | cloudnull, +w for my patches please dude. | 21:44 |
cloudnull | drifterza: I'm actually looking this over now. | 21:47 |
drifterza | k cool | 21:47 |
michaelgugino | automagically: I replied to your comment on my patch set. | 21:47 |
*** gouthamr has quit IRC | 21:49 | |
*** admin0 has quit IRC | 21:50 | |
*** e-vad has joined #openstack-ansible | 21:50 | |
*** vishwanathj is now known as vishwanathj_ | 21:51 | |
cloudnull | done. drifterza | 21:52 |
cloudnull | nicely done on the trove to the integrated repo PR . | 21:53 |
*** schwicht has joined #openstack-ansible | 21:53 | |
cloudnull | looks good | 21:53 |
drifterza | thanks | 21:53 |
drifterza | will look into doing tempest tests in the near future | 21:53 |
*** jorge_munoz has quit IRC | 21:54 | |
*** hughmFLEXin has quit IRC | 21:56 | |
*** jperry has quit IRC | 22:02 | |
*** schwicht has quit IRC | 22:06 | |
*** fops has quit IRC | 22:07 | |
*** david-lyle has quit IRC | 22:09 | |
*** alij has joined #openstack-ansible | 22:10 | |
*** alij_ has joined #openstack-ansible | 22:11 | |
*** alij has quit IRC | 22:11 | |
*** woodard has joined #openstack-ansible | 22:12 | |
*** schwicht has joined #openstack-ansible | 22:13 | |
*** michaelgugino has quit IRC | 22:14 | |
*** alij_ has quit IRC | 22:16 | |
*** sdake has quit IRC | 22:16 | |
*** mkrish004 has quit IRC | 22:19 | |
openstackgerrit | Nish Patwa(nishpatwa_) proposed openstack/openstack-ansible-ops: Added cinder metric collection and telegraf conntrack plugin. https://review.openstack.org/379138 | 22:22 |
*** woodard has quit IRC | 22:23 | |
*** galstrom is now known as galstrom_zzz | 22:24 | |
*** chris_hultin is now known as chris_hultin|AWA | 22:26 | |
*** adrian_otto has joined #openstack-ansible | 22:26 | |
openstackgerrit | Merged openstack/openstack-ansible-os_nova: Move "--no-binary libvirt-python" to ansible-tests https://review.openstack.org/381045 | 22:29 |
*** sdake has joined #openstack-ansible | 22:33 | |
*** schwicht has quit IRC | 22:34 | |
*** TxGirlGeek has quit IRC | 22:35 | |
*** david-lyle has joined #openstack-ansible | 22:36 | |
openstackgerrit | Merged openstack/openstack-ansible: Add integrated Trove deployment. https://review.openstack.org/374008 | 22:44 |
*** phalmos has quit IRC | 22:45 | |
*** kylek3h has quit IRC | 22:46 | |
openstackgerrit | Merged openstack/openstack-ansible: Adding Trove Horizon dashboard https://review.openstack.org/375251 | 22:47 |
openstackgerrit | Merged openstack/openstack-ansible-os_ironic: Move to non stand-alone ironic https://review.openstack.org/380438 | 22:53 |
*** cathrichardson has quit IRC | 22:55 | |
*** weezS has quit IRC | 22:58 | |
*** poopcat has quit IRC | 23:03 | |
*** fops has joined #openstack-ansible | 23:07 | |
*** klamath has quit IRC | 23:12 | |
*** fops has quit IRC | 23:13 | |
*** schwicht has joined #openstack-ansible | 23:15 | |
*** adrian_otto has quit IRC | 23:17 | |
*** adrian_otto has joined #openstack-ansible | 23:20 | |
*** alij has joined #openstack-ansible | 23:20 | |
*** thorst has joined #openstack-ansible | 23:20 | |
*** alij has quit IRC | 23:21 | |
*** pmannidi has joined #openstack-ansible | 23:21 | |
*** alij has joined #openstack-ansible | 23:21 | |
*** alij has quit IRC | 23:26 | |
*** thorst has quit IRC | 23:28 | |
*** thorst has joined #openstack-ansible | 23:28 | |
*** markvoelker has quit IRC | 23:29 | |
*** jorge_munoz has joined #openstack-ansible | 23:29 | |
*** fguillot has quit IRC | 23:31 | |
*** thorst has quit IRC | 23:31 | |
*** thorst has joined #openstack-ansible | 23:32 | |
*** jorge_munoz has quit IRC | 23:33 | |
*** jamielennox is now known as jamielennox|away | 23:35 | |
*** david-lyle has quit IRC | 23:44 | |
*** v1k0d3n has joined #openstack-ansible | 23:45 | |
*** jamielennox|away is now known as jamielennox | 23:48 | |
*** v1k0d3n has quit IRC | 23:50 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!