clarkb | jogo: tempest just started | 00:03 |
---|---|---|
openstackgerrit | Joe Gordon proposed openstack-infra/elastic-recheck: Add query for bug 1394408 https://review.openstack.org/135789 | 00:08 |
* jogo watches | 00:08 | |
*** Sukhdev has quit IRC | 00:08 | |
*** rbak has quit IRC | 00:08 | |
*** oomichi_ has joined #openstack-qa | 00:09 | |
mtreinish | jogo: 2014-11-20 00:08:42.401 | {0} tempest.api.compute.test_live_block_migration.LiveBlockMigrationTestXML.test_live_block_migration [5.044152s] ... FAILED | 00:10 |
mtreinish | at least it tried :) | 00:11 |
jogo | mtreinish: mtreinish woot! | 00:11 |
clarkb | there is a non block one too ya? | 00:11 |
jogo | hmm SKIPPED: Block Live migration not available | 00:12 |
mtreinish | jogo: that's the isci one | 00:12 |
mtreinish | clarkb: no, I think they're all called block migration | 00:12 |
mtreinish | clarkb: when I looked at the code there were 2 tests both do live migrate on a server | 00:13 |
mtreinish | the only difference is that one has a volume attached | 00:13 |
jogo | mtreinish: ohh {USE_BLOCK_MIGRATION_FOR_LIVE_MIGRATION | 00:13 |
jogo | well if it doesn't work without a volume it won't work with most likely | 00:14 |
mtreinish | jogo: yeah, that's the other option, but I figured you only wanted to try the first one without a volume | 00:14 |
openstackgerrit | Merged openstack-infra/elastic-recheck: Add query for bug 1394408 https://review.openstack.org/135789 | 00:14 |
mtreinish | that's all your tempest patch was turning on | 00:14 |
jogo | yeah | 00:14 |
jogo | now I just have to wait for the logs to be collected | 00:14 |
*** adalbas has quit IRC | 00:14 | |
jogo | if it worked on the first try I wouldn't have believed it anyway | 00:15 |
*** dims has quit IRC | 00:15 | |
*** dims has joined #openstack-qa | 00:15 | |
*** yamamoto has joined #openstack-qa | 00:16 | |
*** jecarey has quit IRC | 00:18 | |
*** openstackrecheck has quit IRC | 00:20 | |
*** openstackrecheck has joined #openstack-qa | 00:20 | |
*** wuhg has joined #openstack-qa | 00:20 | |
*** yamamoto has quit IRC | 00:20 | |
*** gmann__ has quit IRC | 00:21 | |
*** gmann has joined #openstack-qa | 00:22 | |
oomichi_ | mtreinish: hi, can you take a look at https://review.openstack.org/#/c/135482/ ? | 00:23 |
jogo | mtreinish: I am a little surprised at how fast that test failed actually | 00:23 |
jogo | 5 seconds | 00:23 |
clarkb | jogo: failing fast is our speciality | 00:23 |
jogo | clarkb: well we fail fast and slow and randomly | 00:24 |
jogo | clarkb: if we just failed fast that would be good | 00:24 |
jogo | and fail quitely | 00:24 |
mtreinish | oomichi_: sure, one sec | 00:24 |
mtreinish | jogo: it probably means it never did anything | 00:24 |
mtreinish | like it failed trying to do the first thing | 00:25 |
oomichi_ | mtreinish: thanks! | 00:25 |
jogo | mtreinish: yeah :/ | 00:26 |
*** adalbas has joined #openstack-qa | 00:26 | |
mtreinish | oomichi_: +A | 00:28 |
openstackgerrit | Ian Wienand proposed openstack-dev/devstack: only use lsb_release for distro tags https://review.openstack.org/117104 | 00:29 |
mtreinish | oomichi_: I'll trade you: https://review.openstack.org/135115 | 00:29 |
mtreinish | jogo: http://paste.openstack.org/show/134987/ | 00:30 |
mtreinish | it's a configuration thing | 00:31 |
mtreinish | clarkb: ^^^ | 00:31 |
mtreinish | to do live migration they need shared storage | 00:31 |
*** dmorita has joined #openstack-qa | 00:32 | |
clarkb | the VM disks need to be on the same fs? | 00:32 |
clarkb | so that would be a thin gwe would need to add to the multinode setup. nfs/ceph/bootfromvolume? | 00:32 |
jogo | clarkb: AFAIK that isn't actually true | 00:32 |
mtreinish | clarkb: iirc there are 2 ways to do, shared storage for the vm disks is the recommended | 00:33 |
jogo | clarkb: but I have to double check | 00:33 |
mtreinish | there is also a libvirt mechanism to do it which is turned off by default because it's flaky | 00:33 |
mtreinish | I think, that might just be out of date info | 00:33 |
mtreinish | clarkb: if we need it, I'd probably just use nfs it would be the simplest to do I think | 00:34 |
clarkb | except we already have cinder | 00:34 |
mtreinish | clarkb: but can we have a volume shared between 2 servers? | 00:35 |
mtreinish | I thought you could only have 1 attach at a time | 00:35 |
jogo | mtreinish: AFAIK we still need a test to do live migration of a boot from volume server | 00:35 |
clarkb | iscsid runs on one host and serves the volume to the compute node | 00:35 |
mtreinish | clarkb: ah, ok | 00:35 |
clarkb | so its shared in that its all on the primary node | 00:36 |
clarkb | and you run off of iscsi | 00:36 |
jogo | clarkb: I think the non shared storage use case is more interesting to test | 00:36 |
clarkb | which may be terrible :) | 00:36 |
*** shashankhegde has quit IRC | 00:37 | |
*** shashankhegde has joined #openstack-qa | 00:37 | |
oomichi_ | mtreinish: yeah, I know. the code seems nice, but the commit message is long and difficult for me :-( | 00:38 |
oomichi_ | mtreinish: I prefer the code, +A | 00:38 |
* jogo reads http://docs.openstack.org/trunk/config-reference/content/section_configuring-compute-migrations.html | 00:39 | |
clarkb | jogo: how does the non shared stoarge migration work? | 00:40 |
*** gfidente has quit IRC | 00:40 | |
mtreinish | clarkb: IIRC it's a libvirt thing | 00:40 |
clarkb | ya so we can do the volume backed thing right now | 00:40 |
clarkb | which is probably a reasonable place to start | 00:41 |
*** yamamoto has joined #openstack-qa | 00:41 | |
mtreinish | jogo, clarkb: http://docs.openstack.org/trunk/config-reference/content/section_configuring-compute-migrations.html#true-live-migration-kvm-libvirt | 00:42 |
mtreinish | there is also a third option if we were running xenserver | 00:42 |
mtreinish | which looking at the name of the test is probably it's origin | 00:42 |
jogo | mtreinish: yeah looks like I need another nova.conf | 00:43 |
mtreinish | clarkb: heh, "The Compute service does not use libvirt's live migration by default because there is a risk that the migration process never ends. This can happen if the guest operating system dirties blocks on the disk faster than they can migrated." | 00:43 |
jogo | gonna poke tripleo folks | 00:43 |
mtreinish | yeah, I think shared storage is the way to go at first | 00:43 |
mtreinish | we don't need more flakiness in the gate | 00:44 |
clarkb | no volume backed | 00:44 |
clarkb | we already have that today | 00:44 |
clarkb | and don't need to change anything | 00:44 |
clarkb | just boot a cirros node off of a cinder volume | 00:44 |
clarkb | migrate node | 00:44 |
mtreinish | clarkb: that would be a new test case I think, where it's a boot from volume instance started | 00:45 |
mtreinish | then we run migrate on that | 00:45 |
clarkb | sure, but it requires no changes to the underlying infrastructure | 00:46 |
clarkb | I think new test case is easier than spin up nfs/ceph on nodes and make shared storage for libvirt | 00:46 |
mtreinish | clarkb: true :) | 00:46 |
mtreinish | jogo: ^^^ want to write it? | 00:46 |
clarkb | but I may be overestimating time to add that atop the multinode stuff | 00:46 |
jogo | block_migration_flag=VIR_MIGRATE_LIVE,VIR_MIGRATE_TUNNELLED,VIR_MIGRATE_PEER2PEER,VIR_MIGRATE_NON_SHARED_INC,VIR_MIGRATE_UNDEFINE_SOURCE | 00:46 |
jogo | is what tripleo tries | 00:46 |
jogo | I am just gonna try that | 00:46 |
jogo | and not the shared storage mode | 00:47 |
mtreinish | jogo: yeah, there's a nice warning in the docs about intermittent failures around doing that | 00:48 |
mtreinish | saying it cause things to hang | 00:48 |
clarkb | what coupld possibly go wrong :) | 00:48 |
clarkb | though with cirros its probably not a big problem | 00:48 |
jogo | mtreinish: do you remember how to add arbitrary nova.conf settings in devstack | 00:48 |
jogo | clarkb: that is what I am thinking | 00:48 |
clarkb | since it runs not a lot of things to write to memory/disk | 00:48 |
mtreinish | jogo: I think you put it in extras something | 00:50 |
mtreinish | there used to be a separate option for it but that went away | 00:50 |
mtreinish | dtroyer: ^^^ are you around | 00:50 |
mtreinish | jogo: I know we go out of our way not to do it in devstack-gate | 00:50 |
jogo | post-config|$NOV_CONF | 00:50 |
jogo | found it I think | 00:51 |
jogo | mtreinish: makes sense, but want to do it quick and dirty as POC first | 00:51 |
clarkb | ya do it and see what happens | 00:51 |
mtreinish | jogo: yeah I agree, I was just saying that because there won't be any examples there | 00:51 |
jogo | although I can use clarkb's hack to include a different devstack branch | 00:51 |
*** hemna is now known as hemnafk | 00:52 | |
mtreinish | clarkb: what do you think: http://cacti.openstack.org/cacti/graph.php?action=view&local_graph_id=2385&rra_id=all | 00:52 |
mtreinish | start of an un-ending growth curve? | 00:52 |
clarkb | mtreinish: possibly | 00:52 |
clarkb | more time will tell | 00:52 |
jogo | http://paste.openstack.org/show/134993 | 00:54 |
jogo | is the idea I am thinking of | 00:54 |
jogo | put it over there so its at the bottom of the localrc file | 00:54 |
jogo | http://hackstack.org/x/blog/2013/09/07/devstack-local-config/ | 00:54 |
jogo | found a dtroyer example | 00:55 |
mtreinish | jogo: heh, yeah that's probably right then :) | 00:55 |
openstackgerrit | Yuiko Takada proposed openstack/tempest: Add Ironic driver API tests https://review.openstack.org/124313 | 00:58 |
openstackgerrit | Joe Gordon proposed openstack-infra/devstack-gate: DO NOT MERGE: Dirty hack to enable non shared storage live migration https://review.openstack.org/135802 | 00:58 |
openstackgerrit | Joe Gordon proposed openstack-infra/devstack-gate: DO NOT MERGE: Dirty hack to enable non shared storage live migration https://review.openstack.org/135802 | 01:00 |
*** suneelb has quit IRC | 01:01 | |
jogo | and running | 01:02 |
*** zz_dimtruck is now known as dimtruck | 01:02 | |
*** yamamoto has quit IRC | 01:08 | |
*** yamamoto has joined #openstack-qa | 01:09 | |
*** yamamoto has quit IRC | 01:10 | |
bknudson | ok, time to look at why gate-tempest-dsvm-src-python-keystoneclient-icehouse fails for https://review.openstack.org/#/c/135127/ | 01:12 |
bknudson | http://logs.openstack.org/27/135127/2/check/gate-tempest-dsvm-src-python-keystoneclient-icehouse/20f49d5/logs/devstacklog.txt.gz says | 01:12 |
bknudson | 2014-11-20 00:04:48.502 | 'suds-jurko' is not a global requirement but it should be,something went wrong | 01:12 |
bknudson | what does that mean. | 01:12 |
clarkb | bknudson: because oslo.vmware didn't have a stbale branch or something | 01:12 |
clarkb | its been fixed I am told | 01:12 |
mtreinish | bknudson: https://bugs.launchpad.net/oslo.vmware/+bug/1394408 | 01:13 |
clarkb | but you were getting master of oslo.vmware which needed a req that wasn't in requirements | 01:13 |
clarkb | jogo: your bash is bad | 01:13 |
* clarkb calls it a day with that :) | 01:13 | |
bknudson | clarkb: thanks! | 01:14 |
mtreinish | clarkb: oslo.vmware just added a new req which wasn't in icehouse global req. Icehouse devstack still installs it from git so icehouse devstack blew up because it wasn't in global-req on icehouse | 01:14 |
*** JayJ has quit IRC | 01:14 | |
mtreinish | I don't think libs should have stable branches | 01:15 |
bknudson | the fix for https://bugs.launchpad.net/oslo.vmware/+bug/1394408 was merged Nov 12, and it's still failing | 01:17 |
bknudson | ok, there's a stable/icehouse version too that merged not long ago... https://review.openstack.org/#/c/135692/ | 01:18 |
mtreinish | bknudson: jogo just pasted the wrong link in the bug | 01:18 |
mtreinish | yeah that;s the right link | 01:18 |
*** dims has quit IRC | 01:24 | |
*** dims has joined #openstack-qa | 01:24 | |
*** achanda has quit IRC | 01:25 | |
jogo | bknudson: woops thanks | 01:28 |
* jogo looks | 01:28 | |
*** melwitt has quit IRC | 01:28 | |
bknudson | looks like all requirements changes in stable/juno are failing... 2014-11-18 07:05:46.642 | 'oslo.middleware' is not in global-requirements.txt | 01:29 |
* jogo watches tempest run yet again in https://jenkins03.openstack.org/job/check-tempest-dsvm-aiopcpu/8/console | 01:29 | |
jogo | bknudson: oh that sounds fun | 01:29 |
jogo | mtreinish: we can't win | 01:29 |
mtreinish | bknudson: I'm done for the night, I pass the torch to you... :) | 01:29 |
mtreinish | jogo: 2014-11-20 01:30:32.308 | {3} tempest.api.compute.test_live_block_migration.LiveBlockMigrationTestJSON.test_live_block_migration [5.363019s] ... FAILED | 01:31 |
jogo | very quick again | 01:31 |
mtreinish | yeah it probably means it failed fast on the live migration again | 01:31 |
jogo | mtreinish: yeah must be missing another conf option | 01:31 |
bknudson | https://review.openstack.org/#/c/135807/ is a change to add oslo.middleware to global-requirements.txt in stable/juno... see if it works. | 01:34 |
bknudson | oh, looks like https://review.openstack.org/#/c/134727/ was supposed to fix that problem. | 01:37 |
openstackgerrit | Merged openstack-dev/devstack: use a more common rst header hiearchy https://review.openstack.org/135252 | 01:45 |
openstackgerrit | Merged openstack/tempest: Move common compute clients to the setting method https://review.openstack.org/135482 | 01:45 |
*** tiantian has joined #openstack-qa | 01:46 | |
openstackgerrit | Ken'ichi Ohmichi proposed openstack/tempest: Add internal methods for setting volume clients https://review.openstack.org/135483 | 01:49 |
openstackgerrit | Ken'ichi Ohmichi proposed openstack/tempest: Add internal methods for setting identity clients https://review.openstack.org/135484 | 01:49 |
*** JayJ has joined #openstack-qa | 01:51 | |
openstackrecheck | Console logs not available after 14:12s for experimental-tempest-dsvm-full-icehouse-dibtest 135768,2,d80494e | 01:55 |
openstackrecheck | openstack/tempest change: https://review.openstack.org/135483 failed because of: gate-grenade-dsvm-partial-ncpu: https://bugs.launchpad.net/bugs/1362528 | 01:55 |
openstackgerrit | Joe Gordon proposed openstack-infra/devstack-gate: Enable live block migration https://review.openstack.org/135768 | 01:56 |
*** yjiang5 is now known as yjiang5_away | 01:57 | |
*** nosnos has joined #openstack-qa | 01:59 | |
adam_g | jogo, RE all this live migration stuff... you're attempting to get libvirt block migration working for the multi host ds setup? | 02:03 |
jogo | adam_g: yup | 02:04 |
jogo | adam_g: have you gotten it working somewhere already? | 02:04 |
adam_g | jogo, i may be of some help, i did some work in the past automating the deployment of that with other things | 02:04 |
jogo | adam_g: nice | 02:05 |
jogo | this is where I am right now https://review.openstack.org/#/c/135768/3 | 02:05 |
jogo | see my comments at the bottom | 02:05 |
jogo | help would be wonderful | 02:05 |
adam_g | jogo, cool, ya. was just looking at that | 02:05 |
adam_g | AFAICS devstack needs to be able to configure libvirtd to listen for connections and setup auth between hosts (either ssh, certificates, etc) | 02:06 |
adam_g | unless thats been added recently? | 02:06 |
jogo | adam_g: we set up ssh keys in devstack gate | 02:07 |
jogo | but don't still need to do libvirtd stuff in devstack | 02:07 |
jogo | adam_g: want to work on that? | 02:07 |
adam_g | jogo, sure one sec | 02:07 |
openstackgerrit | Sean Dague proposed openstack-dev/devstack: add unit tests for GIT* definitions https://review.openstack.org/135711 | 02:07 |
openstackgerrit | Sean Dague proposed openstack-dev/devstack: WIP: test breaking unit tests https://review.openstack.org/135817 | 02:07 |
jogo | adam_g: awesome!! | 02:07 |
openstackgerrit | Merged openstack-infra/elastic-recheck: Omit ironic agent_ssh jobs from 139099 https://review.openstack.org/135770 | 02:08 |
*** dkranz has joined #openstack-qa | 02:19 | |
*** openstackrecheck has quit IRC | 02:20 | |
*** openstackrecheck has joined #openstack-qa | 02:20 | |
openstackgerrit | Adam Gandelman proposed openstack-dev/devstack: Setup libvirt for ssh-based block migration https://review.openstack.org/135821 | 02:27 |
adam_g | jogo, something like that, if memory serves ^ | 02:27 |
jogo | adam_g: you may be missing something as per http://www.sebastien-han.fr/blog/2012/07/12/openstack-block-migration/ | 02:28 |
adam_g | jogo, is there a TLDR for getting my own multi-host devstack up similar to how it is in the gate? | 02:29 |
jogo | adam_g: not sure | 02:29 |
jogo | adam_g: see devstack-gate logs | 02:29 |
jogo | and open patches | 02:29 |
jogo | clarkb: ^ | 02:30 |
adam_g | yeah | 02:30 |
adam_g | figured as much :) | 02:30 |
sdague | adam_g: not really yet, some of that should come back into devstack, and we should mirror it in devstack-vagrant. But it's still such a WIP the replication is ... not there yet | 02:30 |
jogo | nice tempest.api.compute.test_live_block_migration.LiveBlockMigrationTestXML.test_live_block_migration [8.023399s] ... FAILED | 02:30 |
jogo | took longer to fail this time | 02:30 |
clarkb | ya most of it isnt merged in any way yet either | 02:31 |
sdague | jogo: so... did you figure out if live migration is actually even possible on soft qemu? | 02:31 |
*** Longgeek has joined #openstack-qa | 02:33 | |
*** X019 has quit IRC | 02:33 | |
adam_g | jogo, i think whats in that patch is enough, at least it gets me a devstack that i can 'virsh -c qemu+ssh://$foo/system' to a devstack on $foo, which is all thats needed by libvirt for migration | 02:34 |
adam_g | im not sure what user nova will attempt to use for ssh tho | 02:34 |
adam_g | i seem to remember having to allow root logins across systems | 02:34 |
adam_g | via ssh keys with no passphrase | 02:34 |
jogo | sdague: yes, mikal told me so | 02:35 |
sdague | jogo: ... but has it worked? | 02:36 |
jogo | sdague: he said it worked, he tried it | 02:36 |
sdague | adam_g: for devstack is there a reason to make that optional? | 02:36 |
adam_g | sdague, i suppose it could be the default, sure. | 02:39 |
*** Longgeek has quit IRC | 02:40 | |
*** oomichi_ has quit IRC | 02:41 | |
sdague | adam_g: yeh, seems like us just turning on live migration facilities in devstack is valid, because it's a dev env. It shouldn't hurt anything else, right? | 02:41 |
jogo | sdague: yeah that logic works for me | 02:41 |
adam_g | sdague, yea | 02:43 |
sdague | everything time you add a config var to devstack, it kills ... something cute | 02:43 |
sdague | sleep deprevation prevents me from figuring out a funny at the end of that | 02:44 |
mtreinish | sdague: any idea on what the rough count of config options is in devstack? | 02:44 |
openstackgerrit | Merged openstack/tempest: Fix inheritance for BaseTestCase https://review.openstack.org/135115 | 02:44 |
sdague | can I discount neutron? | 02:44 |
mtreinish | lifeless: ^^^ | 02:44 |
mtreinish | sdague: heh, sure thats probably 2x everything | 02:45 |
openstackgerrit | Merged openstack/tempest: Enable cinder v2 api for volume type extra specs tests https://review.openstack.org/135211 | 02:45 |
openstackgerrit | Merged openstack/tempest: Enable v2 api for volume and snapshot tests https://review.openstack.org/135190 | 02:45 |
*** X019 has joined #openstack-qa | 02:46 | |
sdague | yeh... too many | 02:46 |
*** dimtruck is now known as zz_dimtruck | 02:47 | |
mtreinish | sdague: so the tempest fix landed should I propose the unpin for testtool 1.4.0? | 02:47 |
sdague | mtreinish: sure, the lastest is still 1.4.0 right? | 02:47 |
sdague | so it will test if it works | 02:47 |
mtreinish | sdague: yeah, 1.4.0 is still the latest | 02:48 |
sdague | go for it | 02:48 |
sdague | we'll know in a few minutes if it was the right fix | 02:48 |
mtreinish | lifeless said that 1.5.0 is pending and that will bring tracebacks on discovery failures | 02:48 |
openstackgerrit | Adam Gandelman proposed openstack-dev/devstack: Setup libvirt for ssh-based block migration https://review.openstack.org/135821 | 02:48 |
sdague | mtreinish: ok, but lets do this thing to test that it actually fixes the 1.4.0 issue | 02:49 |
lifeless | o/ | 02:50 |
lifeless | yes 1.5.0 is pending | 02:50 |
lifeless | I was trying to get rish tracebacks in | 02:50 |
lifeless | but I think it will take a few more days | 02:50 |
lifeless | so I'll cut 1.5.0 this afternoon probably. | 02:50 |
mtreinish | sdague: https://review.openstack.org/135828 | 02:50 |
sdague | mtreinish: cool. If the tempest tests pass I'll +A it | 02:51 |
*** shashankhegde has quit IRC | 02:51 | |
sdague | dtroyer: also, the devstack unit test now has a staleness unit test for the ALL_LIBS bit | 02:54 |
*** dkranz has quit IRC | 02:55 | |
jogo | adam_g: if that passes tests +1 from me | 02:56 |
openstackgerrit | Eli Qiao proposed openstack/tempest: Add Conflict exception in test_floating_ips_actions case https://review.openstack.org/131364 | 02:58 |
jogo | mtreinish: so this isn't what I expected http://logs.openstack.org/68/135768/3/experimental/check-tempest-dsvm-aiopcpu/7b26557/logs/testr_results.html.gz | 03:00 |
jogo | clarkb: ^ | 03:00 |
jogo | NoLiveMigrationForConfigDriveInLibVirt: Live migration of instances with config drives is not supported in libvirt unless libvirt instance path and drive data is shared across compute nodes. | 03:03 |
clarkb | nice | 03:03 |
lifeless | oh yeah | 03:03 |
lifeless | use metadata server | 03:03 |
lifeless | its better | 03:03 |
clarkb | tl;dr we should just not support live migration is what I am hearing >_> | 03:03 |
lifeless | :) | 03:03 |
clarkb | except for when it doesn't work | 03:03 |
clarkb | which is almost always | 03:03 |
lifeless | wat | 03:04 |
* clarkb shakes fist at being unable to boot centos nodes on hpcloud | 03:04 | |
clarkb | lifeless: I cannot boot centos nodes in hpcloud at all with my account | 03:04 |
clarkb | flat doesn't work becaues metadata server fails | 03:04 |
clarkb | so no ssh key and things | 03:04 |
clarkb | its fun | 03:04 |
lifeless | in what way does it fail ? | 03:04 |
clarkb | cloud init can't get my key | 03:04 |
jogo | http://logs.openstack.org/68/135768/3/experimental/check-tempest-dsvm-aiopcpu/7b26557/logs/10.208.167.5/screen-n-cpu.txt.gz?level=TRACE#_2014-11-20_02_29_09_748 | 03:04 |
clarkb | it times out talking to the metadata server over the 169 address | 03:04 |
lifeless | clarkb: because centos is breaking 169.254.169.254 | 03:04 |
clarkb | lifeless: no I think hpcloud is breaking it | 03:05 |
clarkb | seen it on all OSes but centos is consistent | 03:05 |
mtreinish | jogo: is there an aiopcpu postgres job? | 03:05 |
clarkb | I have filed a couple bugs about it with hpcloud before I gave up | 03:05 |
lifeless | clarkb: is your centos image dib built or ? | 03:05 |
clarkb | no its hpcloud provided | 03:05 |
clarkb | but it failed on precise and trusty too | 03:05 |
clarkb | just not as reliably | 03:06 |
clarkb | (again using the base images provided to me not dib) | 03:06 |
lifeless | so | 03:06 |
jogo | mtreinish: no :/ | 03:06 |
clarkb | dib sort of solve sthis because you don't need metadata server with dib | 03:06 |
jogo | but this is devstack-gate patches | 03:06 |
lifeless | there's a service, name escaping me rigt now | 03:06 |
clarkb | since ou start with an image prekeed | 03:06 |
jogo | so we can change that there | 03:06 |
lifeless | which mangles the 169.254.169.254 address | 03:06 |
lifeless | route | 03:06 |
mtreinish | jogo: yeah flip that to postgres just to test if it works with the metadata server | 03:06 |
lifeless | clarkb: do you get a route dump in your console? | 03:07 |
sdague | mtreinish: or just set the metadata server flag | 03:07 |
lifeless | clarkb: found it | 03:07 |
mtreinish | sdague: oh, yeah | 03:07 |
lifeless | clarkb: http://docs.openstack.org/image-guide/content/centos-image.html | 03:07 |
lifeless | Disable the zeroconf route | 03:08 |
lifeless | For the instance to access the metadata service, you must disable the default zeroconf route: | 03:08 |
*** dims has quit IRC | 03:08 | |
lifeless | # echo "NOZEROCONF=yes" >> /etc/sysconfig/network | 03:08 |
lifeless | clarkb: worth checking that that is setup witin the image | 03:08 |
clarkb | lifeless: that requires getting on the node... | 03:08 |
clarkb | which I cannot do... | 03:08 |
lifeless | clarkb: you can't pull it out of glance? | 03:08 |
mtreinish | jogo: I guess we'll need shared storage if we want to do migration with config drive | 03:08 |
clarkb | lifeless: maybe? | 03:08 |
clarkb | in any case I don't care anymore | 03:08 |
*** dims has joined #openstack-qa | 03:08 | |
clarkb | I just use rax | 03:08 |
sdague | which is why metadata server is kind of bonghits, because it relies on much magic | 03:08 |
clarkb | which doesn't metadata server | 03:08 |
lifeless | mtreinish: or fix config-drive :) | 03:08 |
*** adalbas has quit IRC | 03:08 | |
jogo | mtreinish: maybe we can get ceph set up | 03:09 |
jogo | mtreinish: as that is a common setup | 03:09 |
jogo | where we == not us but ceph folks | 03:09 |
sdague | jogo: you seriously don't want to have to bite that off with this | 03:09 |
lifeless | sdague: mmm it has different dependencies | 03:09 |
mtreinish | jogo: I would just do nfs I think it would be a lot easier | 03:09 |
jogo | sdague: as a follow up down the road | 03:09 |
mtreinish | jogo: or you can try clarkb's iscsid cinder idea | 03:09 |
sdague | so, for this just turn on md | 03:09 |
lifeless | sdague: but I don't think bonghits is a good summary - its in *extremely* widespread use, as I know you know. | 03:09 |
sdague | lifeless: sure :) | 03:11 |
openstackgerrit | Joe Gordon proposed openstack-infra/devstack-gate: DO NOT COMMIT disable config drive https://review.openstack.org/135831 | 03:11 |
clarkb | using the iso mount thing (is that config drive?) with cloud init seems to be most reliable at least when I derp with things locally | 03:11 |
jogo | clarkb: yeah iso is configdrive | 03:11 |
clarkb | I kinda wish that was more common but I guess creating lots of little file sfor mounting is a pita | 03:11 |
sdague | clarkb: yeh, it's a ton more reliable, which is why we defaulted it | 03:11 |
jogo | ok hacked off config drive | 03:12 |
*** krtaylor has quit IRC | 03:12 | |
jogo | mtreinish: long run I think we need a live migrate test wtih and without shared file system | 03:13 |
jogo | but that is long term | 03:13 |
jogo | rerunning tempest with no config drive, lets see if this fails correctly :) | 03:14 |
clarkb | jogo: so this is me being pragmatic. why do we need to support 3 different forms of live migration? | 03:14 |
jogo | clarkb: because legacy | 03:14 |
jogo | clarkb: and 3? | 03:14 |
clarkb | seriously would it be such a terrible thing if we put our foot down and made things sane | 03:14 |
jogo | clarkb: in short I agree nova supports too much | 03:14 |
clarkb | jogo: shared fs, volume backed, libvirt block | 03:14 |
jogo | clarkb: someone wants nova to support PXE booting | 03:14 |
*** HenryG has quit IRC | 03:15 | |
mtreinish | clarkb: to be fair we don't have a volume backed test in tempest yet | 03:15 |
lifeless | jogo: I'm one of those people | 03:15 |
jogo | clarkb: and someone filed a bug saying volueme backed is broken | 03:15 |
sdague | ok, train nearly in station. See folks on the flip side. | 03:15 |
jogo | lifeless: you want nova PXE boot? | 03:15 |
jogo | sdague: o/ | 03:15 |
lifeless | jogo: yes | 03:15 |
jogo | lifeless: why :( | 03:15 |
jogo | lifeless: don't we have enough ways to boot things | 03:16 |
lifeless | jogo: so we can test a lot of tripleo in the regular gate vms | 03:16 |
jogo | lifeless: boot a image with iPXE on it or something | 03:16 |
lifeless | jogo: erm, rephrease. So we can use VMs from the regular quota of cloud machines. | 03:16 |
lifeless | jogo: thats a possibility. See the quintupleo spec :) | 03:17 |
jogo | lifeless: why not do this: http://kimizhang.wordpress.com/2013/08/26/create-pxe-boot-image-for-openstack/ | 03:17 |
mtreinish | clarkb: are you seeing any tracebacks in the subunit gearman worker log, I'm really surprised that a single worker is keeping up at all | 03:17 |
mtreinish | my sql queries suck and should be slower than this... | 03:17 |
jogo | mtreinish: haha when things appear to work you are suspicious | 03:17 |
clarkb | mtreinish: I haven't looked. sort of afraid to this late in the day :) | 03:18 |
mtreinish | clarkb: sure no worries | 03:18 |
lifeless | jogo: well, we're going to then drop a differen timage on the machine | 03:18 |
lifeless | jogo: and then reboot it | 03:18 |
lifeless | jogo: and it should pxe boot again. | 03:18 |
*** HenryG has joined #openstack-qa | 03:18 | |
lifeless | jogo: Rackspace already support PXE booting | 03:18 |
lifeless | jogo: so this is arguably just delta reduction :) | 03:18 |
jogo | lifeless: there is some Pxe support in xen | 03:19 |
lifeless | jogo: and kvm | 03:19 |
jogo | I am fine with adding that support to kvm | 03:19 |
lifeless | jogo: or do you mean the driver? | 03:19 |
jogo | lifeless: yeah the nova drivers | 03:19 |
lifeless | righto | 03:19 |
lifeless | so cool :) | 03:19 |
*** zz_dimtruck is now known as dimtruck | 03:19 | |
jogo | but that isn't what is being proposed right now | 03:19 |
lifeless | whats being proposed right now? | 03:20 |
jogo | lifeless: https://review.openstack.org/133254 | 03:20 |
mtreinish | jogo: I am suspicious, we let it run for a day w/ 4 workers it ate 16GB of mem/swap but kept up with everything | 03:20 |
jogo | wow not bad | 03:21 |
mtreinish | we dropped down to 1, the memory footprint has been sitting at <=.5gb and the gearman queue is still 0 | 03:21 |
mtreinish | granted the load has been light since we restarted things | 03:22 |
jogo | mtreinish: nice | 03:22 |
lifeless | mtreinish: simple schema? | 03:23 |
jogo | anyway my eyes are starting to glaze over, will continue the road to testing live migration without shared storage tomorrow | 03:23 |
jogo | latest job: https://jenkins05.openstack.org/job/check-tempest-dsvm-aiopcpu/7/console | 03:23 |
clarkb | jogo: still expected to fail right? | 03:24 |
lifeless | jogo: https://bugzilla.redhat.com/show_bug.cgi?id=1126268 | 03:24 |
*** nosnos has quit IRC | 03:24 | |
lifeless | jogo: mau be related | 03:24 |
mtreinish | lifeless: more or less, but one table gets very large after a while which really slows down things | 03:25 |
lifeless | jogo: I'm not sure why redhat bugzilla is cloning specs into bugs | 03:25 |
*** krtaylor has joined #openstack-qa | 03:25 | |
*** dimtruck is now known as zz_dimtruck | 03:25 | |
lifeless | mtreinish: how many million rows? | 03:25 |
*** adalbas has joined #openstack-qa | 03:25 | |
mtreinish | I ran it for about 1.5 days locally and it hit ~5.5 million | 03:25 |
lifeless | tiny | 03:26 |
*** yamahata has joined #openstack-qa | 03:26 | |
mtreinish | lifeless: its a row for each individual tempest test executed by ci | 03:26 |
mtreinish | lifeless: heh, it's the biggest table I've worked with :) | 03:27 |
*** rushiagr_away is now known as rushiagr | 03:27 | |
lifeless | jogo: presume you've seen https://bugs.launchpad.net/nova/+bug/1183885 ? | 03:27 |
lifeless | mtreinish: even sqlite can chew that up and spit it out sub-ms | 03:27 |
lifeless | mtreinish: assuming appropriate queries and indices | 03:27 |
clarkb | lifeless: the problem is what it looks like after a dev cycle on a poor trove node | 03:28 |
clarkb | we will find out :) | 03:28 |
mtreinish | lifeless: the 2nd half :) | 03:28 |
*** Longgeek has joined #openstack-qa | 03:36 | |
openstackgerrit | Matthew Treinish proposed openstack/tempest: Unskip test_disassociate_not_associated_floating_ip() https://review.openstack.org/121570 | 03:39 |
openstackgerrit | Matthew Treinish proposed openstack/tempest: Unskip test_list_virtual_interfaces() https://review.openstack.org/121571 | 03:39 |
openstackgerrit | Matthew Treinish proposed openstack/tempest: Unskip test_get_server_diagnostics_by_admin() https://review.openstack.org/121572 | 03:39 |
openstackgerrit | Matthew Treinish proposed openstack/tempest: Unskip test_create_delete_server_group_with_multiple_policies() https://review.openstack.org/121574 | 03:40 |
jogo | clarkb: yeah expecting to fail | 03:40 |
*** Longgeek has quit IRC | 03:41 | |
openstackgerrit | Matthew Treinish proposed openstack-infra/subunit2sql: Flush out the db api documenation https://review.openstack.org/129405 | 03:42 |
jogo | lifeless: odd, that explains things | 03:42 |
openstackgerrit | Matthew Treinish proposed openstack-infra/subunit2sql: Reorganize the python api documentation https://review.openstack.org/129406 | 03:42 |
openstackgerrit | Matthew Treinish proposed openstack-infra/subunit2sql: Add section on the basic data model https://review.openstack.org/129407 | 03:42 |
openstackgerrit | Matthew Treinish proposed openstack/tempest-lib: Add pretty_tox.sh wrapper script to use subunit-trace https://review.openstack.org/129419 | 03:43 |
jogo | lifeless: do you know how to find redhats plan via bugzilla? | 03:43 |
openstackgerrit | Matthew Treinish proposed openstack/tempest: Sync latest log module from oslo-incubator https://review.openstack.org/131601 | 03:44 |
jogo | oh hehe https://bugzilla.redhat.com/buglist.cgi?component=RFEs&product=Red%20Hat%20OpenStack | 03:44 |
jogo | that doesn't look like a manually made list | 03:46 |
*** nosnos has joined #openstack-qa | 03:49 | |
lifeless | jogo: I just googled | 03:49 |
*** yamamoto has joined #openstack-qa | 03:53 | |
*** rushiagr is now known as rushiagr_away | 03:53 | |
*** jgrimm is now known as zz_jgrimm | 03:54 | |
*** patrickeast has quit IRC | 04:05 | |
*** pcrews has quit IRC | 04:16 | |
openstackgerrit | Merged openstack/tempest: Add internal methods for setting volume clients https://review.openstack.org/135483 | 04:24 |
openstackgerrit | Ken'ichi Ohmichi proposed openstack/tempest: Add internal methods for setting identity clients https://review.openstack.org/135484 | 04:26 |
*** zz_dimtruck is now known as dimtruck | 04:27 | |
jgriffith | jogo: clarkb I have to say I'm one of those people that probably don't agree WRT to live-migration restrictions | 04:29 |
jgriffith | jogo: clarkb but we should talk some time over a virtual beer or cofee or whatever | 04:29 |
jgriffith | jogo: clarkb honestly IMO we should just make it "work" | 04:30 |
jgriffith | jogo: clarkb and screw the options/methods, just figure out what needs done and do it | 04:30 |
jgriffith | jogo: clarkb and the number one question from customers as of late is in fact "WTF's up with live-migration in Nova" | 04:31 |
clarkb | ya I thibk it should work | 04:31 |
clarkb | and best way for that is to have one option | 04:32 |
clarkb | that works | 04:32 |
jgriffith | clarkb: +1000 | 04:32 |
clarkb | 3 that dont work is useless | 04:32 |
jgriffith | clarkb: +1000 * 100000000 | 04:32 |
jgriffith | :) | 04:32 |
jgriffith | clarkb: setting aside the whole fact that it confuses the hell out of everybody the way it is now anyway :( | 04:32 |
*** rushiagr_away is now known as rushiagr | 04:33 | |
clarkb | ya I was very confused earlier today | 04:33 |
jgriffith | clarkb: Yay!! | 04:34 |
jgriffith | clarkb: Ok, so next week when I get back and start poking it won't be a waste of time | 04:34 |
clarkb | I have learned so much about openstack doing this multinode stuff :) | 04:35 |
jgriffith | clarkb: Yeah, it's amazing how silo'd I was in my little devstack all in one world :) | 04:35 |
*** dimtruck is now known as zz_dimtruck | 04:37 | |
*** Longgeek has joined #openstack-qa | 04:37 | |
*** yamamoto has quit IRC | 04:39 | |
*** Longgeek has quit IRC | 04:42 | |
openstackgerrit | jun xie proposed openstack/tempest: enable cinder v2 api for test_volumes_backup https://review.openstack.org/135179 | 04:56 |
*** yamamoto has joined #openstack-qa | 05:06 | |
*** shashankhegde has joined #openstack-qa | 05:13 | |
openstackgerrit | Merged openstack/tempest: Add config options for XML support in Keystone https://review.openstack.org/126564 | 05:13 |
*** tiantian has quit IRC | 05:18 | |
*** yfried has quit IRC | 05:27 | |
*** achanda has joined #openstack-qa | 05:38 | |
*** abehl has joined #openstack-qa | 05:41 | |
openstackrecheck | openstack/cinder change: https://review.openstack.org/133409 failed gate-grenade-dsvm in the gate queue with an unrecognized error | 05:47 |
*** chandankumar has joined #openstack-qa | 05:52 | |
*** bkopilov has joined #openstack-qa | 05:53 | |
*** Longgeek has joined #openstack-qa | 06:01 | |
*** Longgeek has quit IRC | 06:02 | |
*** Longgeek has joined #openstack-qa | 06:03 | |
*** tiantian has joined #openstack-qa | 06:11 | |
*** bkopilov has quit IRC | 06:12 | |
*** shashankhegde has quit IRC | 06:13 | |
*** yfried has joined #openstack-qa | 06:17 | |
*** oomichi has joined #openstack-qa | 06:22 | |
openstackgerrit | jun xie proposed openstack/tempest: add cinder v2 api tests for volume_services https://review.openstack.org/135860 | 06:22 |
*** sridhar has joined #openstack-qa | 06:23 | |
*** neeti_ has joined #openstack-qa | 06:27 | |
yfried | sergsh_away: ping | 06:28 |
*** harlowja is now known as harlowja_away | 06:31 | |
*** SridharG has joined #openstack-qa | 06:31 | |
*** k4n0 has joined #openstack-qa | 06:33 | |
*** yfried is now known as yfried|afk | 06:40 | |
*** achanda has quit IRC | 06:45 | |
*** yfried|afk is now known as yfried | 06:48 | |
*** nmagnezi has joined #openstack-qa | 06:53 | |
*** afazekas has quit IRC | 06:55 | |
*** pcrews has joined #openstack-qa | 06:55 | |
*** vigneshvar has joined #openstack-qa | 06:55 | |
openstackgerrit | Bharat Kumar Kobagana proposed openstack-dev/devstack: Deploy GlusterFS server https://review.openstack.org/133102 | 06:56 |
*** achanda has joined #openstack-qa | 06:58 | |
*** achanda has quit IRC | 06:58 | |
*** achanda has joined #openstack-qa | 06:59 | |
*** oomichi__ has joined #openstack-qa | 07:01 | |
*** oomichi has quit IRC | 07:02 | |
*** ashishg has joined #openstack-qa | 07:11 | |
*** bkopilov has joined #openstack-qa | 07:12 | |
*** bkopilov has quit IRC | 07:13 | |
openstackgerrit | Merged openstack/tempest: Add internal methods for setting identity clients https://review.openstack.org/135484 | 07:15 |
*** pcrews has quit IRC | 07:16 | |
jogo | jgriffith: we can do real beer and virtual talk | 07:20 |
*** mpaolino has joined #openstack-qa | 07:22 | |
jogo | jgriffith clarkb: I agree nova API is terribad | 07:22 |
*** neeti_ has quit IRC | 07:22 | |
jogo | well openstack APIs | 07:22 |
jogo | woot http://logs.openstack.org/31/135831/1/experimental/check-tempest-dsvm-aiopcpu/1109b15/logs/screen-n-cpu.txt.gz?level=TRACE#_2014-11-20_03_49_01_648 | 07:23 |
jogo | that is the failure that I was looking for | 07:24 |
*** achanda has quit IRC | 07:25 | |
*** achanda has joined #openstack-qa | 07:25 | |
*** achanda has quit IRC | 07:25 | |
*** ajo has joined #openstack-qa | 07:28 | |
jogo | clarkb: yeah we need a devstack-gate patch for host names libvirtError: operation failed: Failed to connect to remote libvirt URI qemu+tcp://devstack-trusty-2-node-rax-dfw-3143753-382.slave.openstack.org/system: Unable to resolve address 'devstack-trusty-2-node-rax-dfw-3143753-382.slave.openstack.org' service '16509': Name or service not known | 07:29 |
jogo | that is fun | 07:30 |
*** e0ne has joined #openstack-qa | 07:30 | |
*** neeti has joined #openstack-qa | 07:41 | |
openstackrecheck | openstack/nova change: https://review.openstack.org/123459 failed gate-nova-pep8 in the gate queue with an unrecognized error | 07:41 |
*** bkopilov has joined #openstack-qa | 07:42 | |
*** afazekas has joined #openstack-qa | 07:45 | |
yfried | oomichi__: ping | 07:50 |
*** jpich has joined #openstack-qa | 07:58 | |
*** markus_z has joined #openstack-qa | 08:02 | |
*** sergsh_away is now known as sergsh | 08:08 | |
*** oomichi_ has joined #openstack-qa | 08:12 | |
*** oomichi__ has quit IRC | 08:12 | |
sergsh | yfried: pong | 08:13 |
yfried | sergsh: about https://review.openstack.org/#/c/135571/ | 08:14 |
*** oomichi_ has quit IRC | 08:14 | |
sergsh | yeah | 08:14 |
yfried | sergsh: you suggested the flag version=Icehose, right? | 08:14 |
*** gszasz has joined #openstack-qa | 08:14 | |
sergsh | actually I suggested icehouse=False, but your option is better :) | 08:15 |
*** b3nt_pin has quit IRC | 08:15 | |
yfried | sergsh: so, where so you think I can push this flag? it doesn't fit on any of the sections. | 08:16 |
*** beagles has joined #openstack-qa | 08:16 | |
sergsh | the "default",I think | 08:16 |
*** beagles is now known as Guest27825 | 08:16 | |
sergsh | because it's related to whole tempest testing | 08:17 |
yfried | sergsh: how do I edit the default? couldn't find it in config.py. maybe I've missed something? | 08:18 |
openstackgerrit | Adam Gandelman proposed openstack-dev/devstack: Setup libvirt for ssh-based block migration https://review.openstack.org/135821 | 08:18 |
openstackgerrit | huangtianhua proposed openstack/tempest: Take InstanceType for launchconfig res creation https://review.openstack.org/135519 | 08:18 |
sergsh | yfried: hmm.. seems like this is populated in TempestConfigPrivate as cfg.CONF.set_default | 08:21 |
sergsh | yfried, then maybe service_available_group? or create another group? | 08:23 |
*** yamamoto has quit IRC | 08:29 | |
yfried | sergsh: I don't want to create a whole new group for that | 08:29 |
sergsh | yfried, was there before such problem? it's weird that only now we met such restriction | 08:31 |
sergsh | i'd suggest service_available_group then, but I'd agree any solution :) | 08:32 |
yfried | sergsh: what do you mean "such problem"? | 08:33 |
sergsh | of tests that should not run/pass on specific releases | 08:33 |
sergsh | because not all changes are backported to previous releases, if I understand right.. | 08:34 |
*** jlanoux has joined #openstack-qa | 08:38 | |
*** luqas has joined #openstack-qa | 08:38 | |
yfried | sergsh: usually this is due to a new feature or change in API so we add a flag (like IPv6 enabled). but this is a bug-fix/behavior-change of regular api | 08:40 |
*** avico has joined #openstack-qa | 08:43 | |
sergsh | yfried, I see, but it's good idea anyway to have such parameter | 08:44 |
yfried | sergsh: not sure, since tempest's supposed to version-less | 08:45 |
*** dmellado has joined #openstack-qa | 08:50 | |
*** sridhar has quit IRC | 08:54 | |
*** jordanP has joined #openstack-qa | 08:58 | |
*** yamamoto has joined #openstack-qa | 08:58 | |
*** safchain has joined #openstack-qa | 08:59 | |
*** pkoniszewski has joined #openstack-qa | 09:02 | |
openstackgerrit | chandan kumar proposed openstack/tempest: Enable cinder v2 api tests in volume quotas https://review.openstack.org/134889 | 09:12 |
*** pkoniszewski has quit IRC | 09:13 | |
*** yfried is now known as yfried|afk | 09:14 | |
*** yfried|afk is now known as yfried | 09:15 | |
*** e0ne has quit IRC | 09:17 | |
*** sridhar has joined #openstack-qa | 09:20 | |
*** Guest27825 has quit IRC | 09:27 | |
*** oomichi has joined #openstack-qa | 09:27 | |
yfried | oomichi: here? | 09:27 |
oomichi | yfried: hi | 09:28 |
yfried | oomichi: sergsh in myself were discussing the skip flag from yesterday | 09:28 |
yfried | oomichi: you guys seem to like version=icehouse | 09:29 |
oomichi | yfried: yes, that seems common way. and maybe we will be able to use it for the other purpose in the future. | 09:29 |
*** mwagner_lap has quit IRC | 09:30 | |
yfried | oomichi: were would you suggest to push it? I can only think of section [default] but I don't know how to add stuff to that | 09:30 |
oomichi | yfried: OK, I will write it on the gerrit. | 09:31 |
*** Longgeek has quit IRC | 09:32 | |
*** beagles has joined #openstack-qa | 09:32 | |
*** beagles is now known as b3nt_pin | 09:33 | |
*** vponomaryov has quit IRC | 09:34 | |
*** vponomaryov has joined #openstack-qa | 09:36 | |
oomichi | yfried: done | 09:37 |
*** yamamoto has quit IRC | 09:39 | |
*** Longgeek has joined #openstack-qa | 09:41 | |
*** pkoniszewski has joined #openstack-qa | 09:44 | |
sergsh | oomichi: maybe it's better to add "max_version" keyword? and then set it to "icehouse", "juno", etc | 09:44 |
*** yfried is now known as yfried|afk | 09:45 | |
*** JayJ has quit IRC | 09:50 | |
oomichi | sergsh: that is an idea, but if some patch are merged icehouse before juno, it cannot cover it. it seems a little overkill for current situation. | 09:51 |
*** JayJ has joined #openstack-qa | 09:51 | |
*** mpaolino has quit IRC | 09:51 | |
*** mpaolino has joined #openstack-qa | 09:52 | |
*** pkoniszewski has quit IRC | 09:53 | |
*** yfried|afk is now known as yfried | 09:53 | |
openstackgerrit | Ken'ichi Ohmichi proposed openstack/tempest: WIP: Add an option for skipping all XML tests https://review.openstack.org/131665 | 09:54 |
openstackgerrit | Ken'ichi Ohmichi proposed openstack/tempest: Move API clients to better places https://review.openstack.org/135895 | 09:54 |
openstackgerrit | Sergey Shnaidman proposed openstack/qa-specs: Add tests for Neutron extensions https://review.openstack.org/135896 | 09:55 |
yfried | oomichi: so you want the flag to be CONF.network.version? seems like it should be in CONF.default.version | 09:56 |
yfried | oomichi: if we want it to be used by other tests as well later | 09:56 |
openstackgerrit | Sergey Shnaidman proposed openstack/qa-specs: Add tests for Neutron extensions https://review.openstack.org/135896 | 09:57 |
*** JayJ has quit IRC | 09:57 | |
*** zz_johnthetubagu is now known as johnthetubaguy | 09:57 | |
sergsh | Please, take a look at blueprint: https://review.openstack.org/#/c/135896/2 | 09:58 |
*** JayJ has joined #openstack-qa | 09:58 | |
oomichi | yfried: that is a nice idea. | 09:58 |
openstackgerrit | jun xie proposed openstack/tempest: add cinder v2 api tests for volume_services https://review.openstack.org/135860 | 09:59 |
openstackgerrit | Ashish Kumar Gupta proposed openstack/tempest: Security Groups API Tests Enhancements https://review.openstack.org/107654 | 10:00 |
openstackgerrit | Yuiko Takada proposed openstack-dev/devstack: add the kernel/ramdisk id to driver_info https://review.openstack.org/135897 | 10:00 |
openstackrecheck | openstack/glance change: https://review.openstack.org/120970 failed because of: gate-glance-docs: https://bugs.launchpad.net/bugs/1298006, gate-tempest-dsvm-full: https://bugs.launchpad.net/bugs/1353962 | 10:01 |
openstackrecheck | openstack/horizon change: https://review.openstack.org/135226 failed gate-tempest-dsvm-full, gate-tempest-dsvm-postgres-full, gate-tempest-dsvm-neutron, gate-tempest-dsvm-neutron-heat-slow, check-devstack-dsvm-cells in the gate queue with an unrecognized error | 10:01 |
*** om has joined #openstack-qa | 10:01 | |
*** JayJ has quit IRC | 10:02 | |
*** JayJ has joined #openstack-qa | 10:03 | |
*** e0ne has joined #openstack-qa | 10:06 | |
*** JayJ has quit IRC | 10:07 | |
*** JayJ has joined #openstack-qa | 10:08 | |
*** aix has joined #openstack-qa | 10:08 | |
openstackgerrit | Ashish Kumar Gupta proposed openstack/tempest: Port API Test Enhancement https://review.openstack.org/135898 | 10:08 |
*** jlibosva has joined #openstack-qa | 10:11 | |
*** mpaolino has quit IRC | 10:12 | |
*** JayJ has quit IRC | 10:12 | |
openstackgerrit | jun xie proposed openstack/tempest: add cinder v2 api tests for volume_services https://review.openstack.org/135860 | 10:12 |
*** JayJ has joined #openstack-qa | 10:13 | |
*** mpaolino has joined #openstack-qa | 10:17 | |
*** salv-orlando_ has joined #openstack-qa | 10:17 | |
*** JayJ has quit IRC | 10:18 | |
openstackgerrit | Rajkumar proposed openstack/tempest: Add Test to Create Port with no security groups https://review.openstack.org/115908 | 10:18 |
*** JayJ has joined #openstack-qa | 10:18 | |
*** salv-orlando has quit IRC | 10:19 | |
*** salv-orlando_ is now known as salv-orlando | 10:19 | |
*** JayJ has quit IRC | 10:23 | |
*** JayJ has joined #openstack-qa | 10:23 | |
*** raja_vedams has joined #openstack-qa | 10:26 | |
raja_vedams | how hot swap a volume | 10:27 |
*** JayJ has quit IRC | 10:28 | |
yfried | oomichi: so how do I add stuff to conf.default | 10:28 |
*** JayJ has joined #openstack-qa | 10:28 | |
raja_vedams | hi JauJ | 10:29 |
*** jordanP has quit IRC | 10:31 | |
*** JayJ has quit IRC | 10:33 | |
*** raja_vedams has quit IRC | 10:33 | |
*** JayJ has joined #openstack-qa | 10:33 | |
oomichi | yfied: that is what I want to know now ;-) one trial is https://review.openstack.org/#/c/131665/, but I'm not sure that is right way.. | 10:34 |
*** JayJ has quit IRC | 10:38 | |
*** JayJ has joined #openstack-qa | 10:38 | |
*** yamamoto has joined #openstack-qa | 10:39 | |
*** jordanP has joined #openstack-qa | 10:39 | |
*** JayJ has quit IRC | 10:43 | |
*** chandankumar has quit IRC | 10:43 | |
*** JayJ has joined #openstack-qa | 10:44 | |
*** yamamoto has quit IRC | 10:44 | |
*** JayJ has quit IRC | 10:48 | |
*** JayJ has joined #openstack-qa | 10:49 | |
*** chandankumar has joined #openstack-qa | 10:50 | |
*** yfried is now known as yfried|afk | 10:52 | |
*** JayJ has quit IRC | 10:53 | |
*** JayJ has joined #openstack-qa | 10:54 | |
*** aalekov has joined #openstack-qa | 10:58 | |
*** JayJ has quit IRC | 10:58 | |
*** tosky has joined #openstack-qa | 10:58 | |
*** JayJ has joined #openstack-qa | 10:59 | |
*** dims has quit IRC | 11:01 | |
*** dims has joined #openstack-qa | 11:01 | |
*** chandankumar has quit IRC | 11:03 | |
*** JayJ has quit IRC | 11:03 | |
*** JayJ has joined #openstack-qa | 11:04 | |
*** chandankumar has joined #openstack-qa | 11:05 | |
openstackgerrit | Joseph Lanoux proposed openstack/tempest: Add ServerListMultiStatus test https://review.openstack.org/135229 | 11:05 |
*** zz_jgrimm is now known as jgrimm | 11:06 | |
openstackgerrit | Ken'ichi Ohmichi proposed openstack/tempest: WIP: Add an option for skipping all XML tests https://review.openstack.org/131665 | 11:07 |
openstackgerrit | Hugh Saunders proposed openstack/tempest: Add timeout for image operations This patch adds a seperate configuration option for image operations. https://review.openstack.org/135911 | 11:08 |
openstackrecheck | Console logs not available after 13:51s for gate-tempest-python27 131665,5,5fa39d4 | 11:08 |
*** JayJ has quit IRC | 11:08 | |
*** JayJ has joined #openstack-qa | 11:09 | |
*** tkammer has joined #openstack-qa | 11:10 | |
*** rfolco has joined #openstack-qa | 11:11 | |
*** aix has quit IRC | 11:11 | |
*** yfried|afk is now known as yfried | 11:13 | |
*** JayJ has quit IRC | 11:14 | |
*** JayJ has joined #openstack-qa | 11:14 | |
*** asselin has quit IRC | 11:16 | |
*** JayJ has quit IRC | 11:19 | |
*** JayJ has joined #openstack-qa | 11:19 | |
*** yfried is now known as yfried|afk | 11:23 | |
*** JayJ has quit IRC | 11:24 | |
*** JayJ has joined #openstack-qa | 11:24 | |
*** yamahata has quit IRC | 11:26 | |
*** rfolco has quit IRC | 11:27 | |
*** yfried|afk is now known as yfried | 11:27 | |
*** rfolco has joined #openstack-qa | 11:28 | |
*** adalbas has quit IRC | 11:28 | |
*** yamahata has joined #openstack-qa | 11:28 | |
*** JayJ has quit IRC | 11:29 | |
*** JayJ has joined #openstack-qa | 11:29 | |
*** tiantian has quit IRC | 11:30 | |
*** JayJ has quit IRC | 11:34 | |
*** JayJ has joined #openstack-qa | 11:34 | |
*** flwang has quit IRC | 11:36 | |
*** Longgeek has quit IRC | 11:39 | |
*** JayJ has quit IRC | 11:39 | |
*** JayJ has joined #openstack-qa | 11:40 | |
*** aix has joined #openstack-qa | 11:42 | |
*** flwang has joined #openstack-qa | 11:43 | |
*** JayJ has quit IRC | 11:44 | |
*** JayJ has joined #openstack-qa | 11:45 | |
openstackgerrit | Udi Kalifon proposed openstack/tempest: Adding support in the client to create users with a default_project_id. Also adding a test to make sure that such users automatically get a token that is scoped to their project - even when requesting an unscoped token. https://review.openstack.org/135922 | 11:46 |
ashishg | why jenkins server is failing for tempest test cases ? | 11:48 |
*** JayJ has quit IRC | 11:49 | |
*** JayJ has joined #openstack-qa | 11:50 | |
openstackgerrit | Abhishek G M proposed openstack/tempest: Test to verify shared attribute of network https://review.openstack.org/124998 | 11:53 |
openstackgerrit | jun xie proposed openstack/tempest: add cinder v2 api tests for volume_services https://review.openstack.org/135860 | 11:54 |
*** nfedotov has joined #openstack-qa | 11:54 | |
*** JayJ has quit IRC | 11:54 | |
*** JayJ has joined #openstack-qa | 11:55 | |
*** mpaolino has quit IRC | 12:05 | |
openstackgerrit | chandan kumar proposed openstack/tempest: Enable cinder v2 api tests in volume quotas https://review.openstack.org/134889 | 12:06 |
chandankumar | gmann, jordanP afazekas https://review.openstack.org/134889 Please have a look | 12:07 |
jordanP | chandankumar, I'd rather wait for jenkins :) | 12:09 |
*** rushiagr is now known as rushiagr_away | 12:15 | |
openstackgerrit | Salvatore Orlando proposed openstack-infra/devstack-gate: Add support for enabling/disabling service extensions https://review.openstack.org/116129 | 12:15 |
*** Longgeek has joined #openstack-qa | 12:19 | |
*** dmorita has quit IRC | 12:23 | |
*** adalbas has joined #openstack-qa | 12:25 | |
*** oomichi has quit IRC | 12:31 | |
sergsh | afazekas, sdague: please, take a look in your time: https://review.openstack.org/#/c/134007/ | 12:34 |
*** chandankumar has quit IRC | 12:37 | |
*** sergsh is now known as sergsh_away | 12:38 | |
*** yamamoto has joined #openstack-qa | 12:41 | |
openstackgerrit | Ken'ichi Ohmichi proposed openstack/tempest: WIP: Add an option for skipping all XML tests https://review.openstack.org/131665 | 12:41 |
*** rushiagr_away is now known as rushiagr | 12:46 | |
*** yamamoto has quit IRC | 12:47 | |
*** nmagnezi has quit IRC | 12:49 | |
*** neeti has quit IRC | 12:52 | |
openstackrecheck | openstack/horizon change: https://review.openstack.org/135251 failed because of: gate-grenade-dsvm: https://bugs.launchpad.net/bugs/1373513 | 12:53 |
*** tkammer has quit IRC | 12:58 | |
*** nmagnezi has joined #openstack-qa | 13:02 | |
*** tosky has quit IRC | 13:04 | |
*** jhenner has quit IRC | 13:04 | |
*** gszasz has quit IRC | 13:05 | |
*** nosnos has quit IRC | 13:05 | |
*** gfidente has joined #openstack-qa | 13:06 | |
*** nosnos has joined #openstack-qa | 13:06 | |
openstackgerrit | Hugh Saunders proposed openstack/tempest: Add timeout for image operations https://review.openstack.org/135911 | 13:07 |
*** nmagnezi has quit IRC | 13:08 | |
*** jhenner has joined #openstack-qa | 13:08 | |
*** gszasz has joined #openstack-qa | 13:12 | |
*** afazekas has quit IRC | 13:13 | |
*** X019 has quit IRC | 13:14 | |
*** tkammer has joined #openstack-qa | 13:14 | |
*** pkoniszewski has joined #openstack-qa | 13:15 | |
openstackgerrit | Louis Taylor proposed openstack-dev/devstack: Install bash completion for glanceclient https://review.openstack.org/135941 | 13:16 |
*** tosky has joined #openstack-qa | 13:19 | |
*** vigneshvar has quit IRC | 13:21 | |
*** zz_dimtruck is now known as dimtruck | 13:21 | |
*** pkoniszewski has quit IRC | 13:22 | |
*** nmagnezi has joined #openstack-qa | 13:23 | |
*** markus_z has quit IRC | 13:25 | |
*** afazekas has joined #openstack-qa | 13:29 | |
openstackgerrit | Sean Dague proposed openstack-dev/devstack: WIP: test breaking unit tests https://review.openstack.org/135817 | 13:32 |
openstackgerrit | Sean Dague proposed openstack-dev/devstack: add unit tests for GIT* definitions https://review.openstack.org/135711 | 13:32 |
openstackgerrit | Sean Dague proposed openstack-dev/devstack: add tool for testing project defs https://review.openstack.org/135948 | 13:32 |
*** dkranz has joined #openstack-qa | 13:33 | |
*** k4n0 has quit IRC | 13:38 | |
*** afazekas has quit IRC | 13:39 | |
*** bkopilov has quit IRC | 13:41 | |
openstackgerrit | Hugh Saunders proposed openstack/tempest: Add timeout for image operations https://review.openstack.org/135911 | 13:43 |
*** salv-orlando has quit IRC | 13:47 | |
*** rfolco has quit IRC | 13:48 | |
*** om has quit IRC | 13:56 | |
*** yfried has quit IRC | 13:56 | |
*** mpaolino has joined #openstack-qa | 13:57 | |
*** abehl has quit IRC | 13:57 | |
*** weshay has joined #openstack-qa | 13:58 | |
*** rushiagr is now known as rushiagr_away | 13:59 | |
*** tkammer has quit IRC | 14:02 | |
*** rushiagr_away is now known as rushiagr | 14:07 | |
*** mkollaro has joined #openstack-qa | 14:11 | |
*** ashishg has quit IRC | 14:13 | |
*** sergsh_ has joined #openstack-qa | 14:13 | |
*** mriedem has joined #openstack-qa | 14:13 | |
*** tkammer has joined #openstack-qa | 14:14 | |
*** afazekas has joined #openstack-qa | 14:15 | |
*** rfolco has joined #openstack-qa | 14:15 | |
*** jhenner has quit IRC | 14:16 | |
*** dustins has joined #openstack-qa | 14:19 | |
*** jhenner has joined #openstack-qa | 14:30 | |
*** rfolco has quit IRC | 14:32 | |
*** nosnos has quit IRC | 14:34 | |
*** masayukig has quit IRC | 14:41 | |
*** toure has joined #openstack-qa | 14:41 | |
*** jecarey has joined #openstack-qa | 14:44 | |
*** masayukig has joined #openstack-qa | 14:45 | |
*** toure has quit IRC | 14:46 | |
*** toure has joined #openstack-qa | 14:47 | |
*** yamamoto has joined #openstack-qa | 14:48 | |
*** markmcclain has joined #openstack-qa | 14:57 | |
*** salv-orlando has joined #openstack-qa | 14:57 | |
*** yamahata has quit IRC | 14:59 | |
*** yamahata has joined #openstack-qa | 15:00 | |
*** markmcclain1 has joined #openstack-qa | 15:00 | |
*** markmcclain has quit IRC | 15:01 | |
*** asselin has joined #openstack-qa | 15:01 | |
*** nmagnezi has quit IRC | 15:02 | |
*** markmcclain1 has quit IRC | 15:05 | |
*** markmcclain has joined #openstack-qa | 15:05 | |
svasheka | hi | 15:10 |
svasheka | is there separate ostf irc channel? | 15:10 |
*** pcrews has joined #openstack-qa | 15:17 | |
*** fmccarthy has joined #openstack-qa | 15:18 | |
openstackgerrit | Alok Kumar Maurya proposed openstack/tempest: Basic networks Scenario Test Enhancements https://review.openstack.org/106620 | 15:19 |
*** yamamoto_ has joined #openstack-qa | 15:20 | |
*** JayJ has quit IRC | 15:21 | |
*** bkopilov has joined #openstack-qa | 15:21 | |
*** yamamoto has quit IRC | 15:21 | |
*** dustins_ has joined #openstack-qa | 15:30 | |
*** rbak has joined #openstack-qa | 15:31 | |
*** dustins has quit IRC | 15:33 | |
*** dpaterson has joined #openstack-qa | 15:34 | |
*** aalekov has quit IRC | 15:34 | |
*** jgrimm is now known as zz_jgrimm | 15:36 | |
*** X019 has joined #openstack-qa | 15:37 | |
*** rbak_ has joined #openstack-qa | 15:42 | |
*** markmcclain has quit IRC | 15:43 | |
openstackgerrit | prabhu murthy proposed openstack/tempest: Added neutron cli test case for fwaas https://review.openstack.org/123633 | 15:44 |
*** rbak has quit IRC | 15:45 | |
*** markmcclain has joined #openstack-qa | 15:46 | |
*** markmcclain has quit IRC | 15:46 | |
*** markmcclain has joined #openstack-qa | 15:47 | |
*** jordanP has quit IRC | 15:48 | |
*** JayJ has joined #openstack-qa | 15:50 | |
*** ajo has quit IRC | 15:57 | |
*** jlibosva has quit IRC | 15:57 | |
*** jlibosva has joined #openstack-qa | 15:59 | |
*** ajo has joined #openstack-qa | 15:59 | |
openstackgerrit | Nikita Koshikov proposed openstack-dev/devstack: Coraid cinder driver added https://review.openstack.org/136034 | 16:00 |
*** dimtruck is now known as zz_dimtruck | 16:04 | |
*** SridharG has left #openstack-qa | 16:04 | |
*** marun has joined #openstack-qa | 16:05 | |
*** zz_dimtruck is now known as dimtruck | 16:06 | |
*** sridhar has quit IRC | 16:07 | |
openstackgerrit | Sean M. Collins proposed openstack-dev/devstack: Documentation: Using Neutron with DevStack https://review.openstack.org/131201 | 16:08 |
*** gszasz has quit IRC | 16:21 | |
*** rbak_ has quit IRC | 16:22 | |
*** rbak_ has joined #openstack-qa | 16:23 | |
*** vigneshvar has joined #openstack-qa | 16:24 | |
*** afazekas has quit IRC | 16:28 | |
*** zz_jgrimm is now known as jgrimm | 16:29 | |
openstackgerrit | Sean M. Collins proposed openstack-dev/devstack: Documentation: Using Neutron with DevStack https://review.openstack.org/131201 | 16:31 |
*** dustins_ has quit IRC | 16:35 | |
*** dustins has joined #openstack-qa | 16:36 | |
*** dustins has quit IRC | 16:37 | |
*** dmellado has quit IRC | 16:41 | |
*** jordanP has joined #openstack-qa | 16:42 | |
openstackgerrit | Merged openstack/tempest: Support for IPv6 tests for API https://review.openstack.org/134007 | 16:42 |
*** Longgeek has quit IRC | 16:45 | |
openstackgerrit | Matthew Treinish proposed openstack-infra/elastic-recheck: Add query for wrapt stable global reqs issues https://review.openstack.org/136049 | 16:45 |
openstackgerrit | Feodor Tersin proposed openstack/tempest: Use standard assertVolumeStatusWait for volume detachment https://review.openstack.org/130262 | 16:46 |
*** mlavalle has joined #openstack-qa | 16:52 | |
*** rfolco has joined #openstack-qa | 16:54 | |
sdague | mtreinish: leaking ip addresses - https://jenkins04.openstack.org/job/gate-tempest-dsvm-full/15755/console ? | 16:54 |
openstackrecheck | openstack/nova change: https://review.openstack.org/134068 failed because of: gate-tempest-dsvm-full: https://bugs.launchpad.net/bugs/1353962, gate-tempest-dsvm-neutron-full: unrecognized error | 16:55 |
sdague | actually, that's probably leaking computes | 16:55 |
mtreinish | sdague: hmm, yeah it's probably servers. I think mriedem was looking at a similar quota issue before | 16:55 |
*** anna_ has joined #openstack-qa | 16:57 | |
*** shashankhegde has joined #openstack-qa | 16:58 | |
openstackgerrit | Joseph Lanoux proposed openstack/tempest: Add ServerUsage test https://review.openstack.org/135222 | 16:59 |
mriedem | fixed ip quotas is a known race bug | 16:59 |
*** X019 has quit IRC | 16:59 | |
mriedem | sdague: http://status.openstack.org/elastic-recheck/gate.html#1353962 | 17:00 |
mriedem | it's an oddball case where the fixed_ips quota is 99 and it can't allocate anything | 17:00 |
*** yfried has joined #openstack-qa | 17:01 | |
sdague | mriedem: is that the delayed dealocation case? | 17:01 |
mriedem | by default fixed_ips quota is -1 (it's the only one that is unlimited by default), then we must be hitting that default quota class test that bumps all default quotas + 100 | 17:01 |
mriedem | not sure what the delayed deallocation case is, | 17:01 |
mriedem | jogo and i were thinking it was some goofy logic in the quotas code with how it handles per-project quotas and unlimited quotas | 17:01 |
*** atiwari has joined #openstack-qa | 17:02 | |
mriedem | as in not handling them correctly | 17:02 |
sdague | it looked like quotas are just super sloppy, then there is a throw hands up and just let the periodic jobs recalc last time I looked | 17:02 |
mriedem | we got additional logging in nova-network when we hit this and it doesn't make much sense | 17:02 |
*** aalekov has joined #openstack-qa | 17:02 | |
mriedem | let me find the log message | 17:02 |
mriedem | it's weird | 17:02 |
sdague | reading all the quota bugs.... quotas are just massively broken in accounting | 17:03 |
mriedem | the quotas code is definitely scary | 17:03 |
mriedem | this http://logs.openstack.org/68/134068/2/gate/gate-tempest-dsvm-full/72da861/logs/screen-n-net.txt.gz?level=TRACE#_2014-11-20_15_12_42_983 | 17:03 |
mriedem | 0 of 99 are in use or are already reserved. | 17:03 |
mriedem | wtf?! | 17:03 |
sdague | qa meeting | 17:04 |
mriedem | but i haven't gotten back into that bug since adding that logging and refactoring some of the db api quota_reserve() code so it's actually readable/unit-testable | 17:04 |
sdague | dkranz ? | 17:04 |
*** markmcclain has quit IRC | 17:06 | |
*** jlanoux has quit IRC | 17:07 | |
openstackgerrit | Jordan Pittier proposed openstack-dev/devstack: Remove unused and deprecated SWIFT_CONFIG_DIR variable https://review.openstack.org/126924 | 17:07 |
*** avico has quit IRC | 17:07 | |
*** yamamoto_ has quit IRC | 17:09 | |
*** rfolco has quit IRC | 17:10 | |
*** markmcclain has joined #openstack-qa | 17:11 | |
openstackgerrit | Merged openstack-infra/elastic-recheck: Add query for wrapt stable global reqs issues https://review.openstack.org/136049 | 17:12 |
sdague | dtroyer: test results are back on https://review.openstack.org/#/c/135711/ if you want to +A | 17:14 |
*** markmcclain has quit IRC | 17:15 | |
dtroyer | sdague: cool, done | 17:15 |
*** yamahata has quit IRC | 17:16 | |
*** sabeen2 has quit IRC | 17:19 | |
*** sabeen1 has joined #openstack-qa | 17:20 | |
*** dkranz has quit IRC | 17:23 | |
*** Viswanath has joined #openstack-qa | 17:24 | |
*** safchain has quit IRC | 17:25 | |
*** jpich has quit IRC | 17:25 | |
*** anna_ is now known as X019 | 17:25 | |
*** shashankhegde has quit IRC | 17:27 | |
*** openstackrecheck has quit IRC | 17:27 | |
*** openstackrecheck has joined #openstack-qa | 17:27 | |
*** Viswanath has quit IRC | 17:27 | |
*** andreykurilin_ has joined #openstack-qa | 17:30 | |
andreykurilin_ | Hi! I'll be happy if somebody review this spec - https://review.openstack.org/#/c/94473/ :) | 17:32 |
*** dimtruck is now known as zz_dimtruck | 17:35 | |
*** dkranz has joined #openstack-qa | 17:36 | |
*** aix has quit IRC | 17:51 | |
mkoderer | sdague: could you give me a hint ... are you refering to https://pypi.python.org/pypi/testresources in https://review.openstack.org/#/c/134472/? | 17:52 |
sdague | mkoderer: yeh | 17:52 |
*** yfried has quit IRC | 17:53 | |
mkoderer | sdague: I saw that we have this lib in requirments.txt... but it is acutally used somewhere? | 17:53 |
mtreinish | mkoderer: it's not there anymore | 17:53 |
mtreinish | it got removed yesterday (because nothing was actually using it) | 17:53 |
*** dustins has joined #openstack-qa | 17:54 | |
mtreinish | but you can add it back if we start using it | 17:54 |
mkoderer | mtreinish: hehe ok... ;) | 17:54 |
*** aalekov has quit IRC | 17:55 | |
mkoderer | mtreinish: I will give it a try... are some other project using it? | 17:55 |
mtreinish | mkoderer: I'm not sure, I don't think so. oslo.db I think has patches up to start using it | 17:56 |
mtreinish | sdague: ^^^ you were looking for examples the other day, did you find any? | 17:57 |
openstackrecheck | openstack/cinder change: https://review.openstack.org/131886 failed because of: gate-grenade-dsvm: https://bugs.launchpad.net/bugs/1394647 | 17:57 |
mkoderer | mtreinish: ok so again pypi beta tester ;) | 17:57 |
sdague | there is an oslo db patch in review | 17:58 |
sdague | honestly, it's a monday thing for me to try to dive into | 17:58 |
*** achanda has joined #openstack-qa | 17:59 | |
mkoderer | sdague: ok, if I have time I will have a look tomorrow | 17:59 |
*** harlowja_away is now known as harlowja | 18:02 | |
*** jlibosva has quit IRC | 18:03 | |
*** shashankhegde has joined #openstack-qa | 18:03 | |
*** e0ne has quit IRC | 18:03 | |
*** suneelb has joined #openstack-qa | 18:04 | |
*** rfolco has joined #openstack-qa | 18:05 | |
*** suneelb has quit IRC | 18:07 | |
adam_g | sdague, i was hoping to flush out this stack of patches for devstack+diskimage-builder. are you still -1 on this? https://review.openstack.org/#/c/122539/ | 18:07 |
*** suneelb has joined #openstack-qa | 18:07 | |
*** gfidente has quit IRC | 18:08 | |
*** yamamoto has joined #openstack-qa | 18:10 | |
*** mriedem has quit IRC | 18:10 | |
sdague | adam_g: so, is there a spec for this? Because I feel like I don't have the whole picture in my head about where it's going. In the narrow, it seems weird to add it there. | 18:10 |
*** krtaylor has quit IRC | 18:13 | |
*** melwitt has joined #openstack-qa | 18:13 | |
adam_g | sdague, there isn't a spec but i can put up a short one explaining. in short, that will enable DIB to be installed and used via facilities lib/dib, which (among other things) will allow it to be installed from release/git (like client libs) and point DIB at non-internet resources for image builds | 18:14 |
*** yamamoto has quit IRC | 18:14 | |
*** jordanP has quit IRC | 18:15 | |
sdague | adam_g: right, so I think all those later things are goodness, I just think it's a thing we can do all the time instead of by flipping a service | 18:15 |
*** krtaylor has joined #openstack-qa | 18:16 | |
sdague | anyway, yeh, sketch down the full story so I can see it, just so we make sure it integrates cleanly | 18:16 |
*** mriedem has joined #openstack-qa | 18:17 | |
*** patrickeast has joined #openstack-qa | 18:17 | |
adam_g | sure | 18:17 |
*** rfolco has quit IRC | 18:22 | |
*** wuhg has quit IRC | 18:23 | |
*** rbak_ has quit IRC | 18:31 | |
*** tkammer has quit IRC | 18:34 | |
*** dkranz has quit IRC | 18:35 | |
*** rbak has joined #openstack-qa | 18:35 | |
*** sergsh_ has quit IRC | 18:37 | |
*** mkollaro has quit IRC | 18:39 | |
*** markmcclain has joined #openstack-qa | 18:43 | |
*** rushiagr is now known as rushiagr_away | 18:45 | |
*** markmcclain has quit IRC | 18:45 | |
*** suneelb has quit IRC | 18:47 | |
*** suneelb has joined #openstack-qa | 18:47 | |
*** markmcclain has joined #openstack-qa | 18:48 | |
*** dkranz has joined #openstack-qa | 18:48 | |
*** mpaolino has quit IRC | 18:48 | |
*** sergsh_ has joined #openstack-qa | 18:49 | |
*** andreykurilin_ has quit IRC | 18:52 | |
*** johnthetubaguy is now known as zz_johnthetubagu | 18:52 | |
*** mkollaro has joined #openstack-qa | 18:53 | |
*** weshay has quit IRC | 18:55 | |
*** markmcclain1 has joined #openstack-qa | 18:56 | |
*** markmcclain has quit IRC | 18:57 | |
*** Viswanath has joined #openstack-qa | 19:00 | |
*** Viswanath has quit IRC | 19:07 | |
*** e0ne has joined #openstack-qa | 19:09 | |
openstackrecheck | openstack/horizon change: https://review.openstack.org/135962 failed because of: check-devstack-dsvm-cells: https://bugs.launchpad.net/bugs/1365046, gate-grenade-dsvm: https://bugs.launchpad.net/bugs/1365046, gate-tempest-dsvm-postgres-full: https://bugs.launchpad.net/bugs/1365046 | 19:11 |
*** salv-orlando has quit IRC | 19:12 | |
*** weshay has joined #openstack-qa | 19:13 | |
*** patrickeast_ has joined #openstack-qa | 19:16 | |
*** patrickeast has quit IRC | 19:17 | |
*** patrickeast_ is now known as patrickeast | 19:17 | |
*** zz_dimtruck is now known as dimtruck | 19:19 | |
mriedem | e-r uncategorized bug % is getting hurt by this super generic issue http://logs.openstack.org/72/123972/2/gate/gate-tempest-dsvm-neutron-heat-slow/be87c42/console.html#_2014-11-17_23_01_33_078 | 19:20 |
mriedem | it's not restricted to heat though | 19:20 |
*** vigneshvar has quit IRC | 19:23 | |
*** Sukhdev has joined #openstack-qa | 19:24 | |
openstackgerrit | Adam Gandelman proposed openstack/qa-specs: Improve diskimage-builder in devstack https://review.openstack.org/136101 | 19:26 |
adam_g | sdague, there ya'r | 19:27 |
*** tosky has quit IRC | 19:29 | |
sdague | adam_g: thanks | 19:31 |
*** rwsu has quit IRC | 19:32 | |
*** rwsu has joined #openstack-qa | 19:33 | |
*** vigneshvar has joined #openstack-qa | 19:35 | |
*** e0ne has quit IRC | 19:37 | |
*** krtaylor has quit IRC | 19:45 | |
*** tkammer has joined #openstack-qa | 19:49 | |
*** mpaolino has joined #openstack-qa | 19:56 | |
*** krtaylor has joined #openstack-qa | 19:58 | |
*** melwitt has quit IRC | 20:06 | |
*** markmcclain1 has quit IRC | 20:08 | |
*** markmcclain has joined #openstack-qa | 20:08 | |
*** melwitt1 has joined #openstack-qa | 20:08 | |
*** melwitt1 is now known as melwitt | 20:09 | |
openstackgerrit | Merged openstack-dev/devstack: Add IPv6 support for tenant data network https://review.openstack.org/87987 | 20:09 |
mtreinish | mriedem: that should actually be the testtools issue which should be fixed now | 20:15 |
mtreinish | twice over actually | 20:15 |
openstackgerrit | Sergey Shnaidman proposed openstack/tempest: DHCPv6 network tests https://review.openstack.org/117458 | 20:15 |
mriedem | mtreinish: but it keeps showing up | 20:16 |
mriedem | is there a bug reported? | 20:16 |
mriedem | we could at least classify it | 20:16 |
mtreinish | mriedem: I don't think so, we kinda just rushed through the fix because everything was failing | 20:16 |
mtreinish | mriedem: at least that was the issue before attrs broke in tempest with testtools 1.4.0 which meant the regex filter had no matches | 20:17 |
mtreinish | we blocked that testtools release and I landed a fix in tempest (we merged the revert of the block this morning) | 20:17 |
mtreinish | mriedem: the way you can tell if it's the same issue is if the full job is running the heat slow tests | 20:18 |
mtreinish | or grenade runs everything... | 20:18 |
*** shashankhegde has quit IRC | 20:20 | |
mriedem | it is a heat slow job, but it's showing up in other jobs also | 20:23 |
mtreinish | mriedem: do you have a link to the other job? | 20:23 |
mriedem | mtreinish: this? http://logs.openstack.org/72/123972/2/gate/gate-tempest-dsvm-neutron-heat-slow/be87c42/console.html#_2014-11-17_23_01_33_078 | 20:24 |
mtreinish | actually grenade would run nothing too, I did the regex match backwards... | 20:24 |
mtreinish | mriedem: that's also heat | 20:24 |
mtreinish | mriedem: actually isn't that the same link? | 20:25 |
mriedem | it is | 20:25 |
mriedem | sec | 20:25 |
openstackgerrit | Adam Gandelman proposed openstack/qa-specs: Improve diskimage-builder in devstack https://review.openstack.org/136101 | 20:28 |
openstackrecheck | Console logs not available after 13:25s for check-tempest-dsvm-full 104930,12,e13a318 | 20:30 |
sergsh_ | who is interested in networking, please review my blueprint about additional tests in tempest: https://review.openstack.org/#/c/135896/ | 20:34 |
*** melwitt has quit IRC | 20:36 | |
*** melwitt has joined #openstack-qa | 20:37 | |
*** vigneshvar has quit IRC | 20:38 | |
*** salv-orlando has joined #openstack-qa | 20:44 | |
*** patrickeast has quit IRC | 20:48 | |
mriedem | mtreinish: sorry, got sidetracked http://goo.gl/k5coaC | 20:49 |
*** patrickeast has joined #openstack-qa | 20:49 | |
mriedem | weird that didn't filter on gate | 20:49 |
*** MasterPiece has joined #openstack-qa | 20:50 | |
*** andreykurilin_ has joined #openstack-qa | 20:50 | |
*** atiwari has quit IRC | 20:50 | |
mtreinish | mriedem: it looks like the only gate tempest job is the heat slow job | 20:52 |
mtreinish | there are a bunch of nova functional and unit tests fails | 20:52 |
mtreinish | and a whole lot of check things (which I'm just going to assume are bad patches) | 20:52 |
mriedem | mtreinish: yeah that's why i was trying to filter out check queue | 20:53 |
mtreinish | mriedem: anyway if you want to open a tempest bug for this | 20:53 |
mtreinish | it was fixed by: https://review.openstack.org/135115 | 20:54 |
mtreinish | but I guess it'll still be on the unclassified list | 20:54 |
mtreinish | s/but/because | 20:54 |
mriedem | mtreinish: ah ok, then that makes more sense | 20:54 |
mriedem | http://status.openstack.org/elastic-recheck/data/uncategorized.html | 20:54 |
mriedem | yeah that's how i was seeing it | 20:54 |
mriedem | i'll get a bug opened and closed and tracked | 20:54 |
mriedem | in e-r i mean | 20:54 |
*** yfried has joined #openstack-qa | 20:59 | |
*** dimtruck is now known as zz_dimtruck | 20:59 | |
dtroyer | mtreinish: in the -specs repo, should we do something to sort out the DevStack specs? Like a subdir or convention of prefixing the files with 'ds-'? Either of those is fine with me | 21:05 |
*** shashankhegde has joined #openstack-qa | 21:06 | |
*** weshay has quit IRC | 21:08 | |
*** e0ne has joined #openstack-qa | 21:10 | |
*** zz_dimtruck is now known as dimtruck | 21:10 | |
*** yamamoto has joined #openstack-qa | 21:12 | |
mtreinish | dtroyer: if you want to do a subdir that's fine, I was thinking of just adding a section to the template on projects it touches | 21:13 |
mtreinish | I'd like to keep the rst files matching the lp bp name though so probably not a prefix | 21:13 |
dtroyer | mtreinish: the audiences don't totally overlap, and even if they did I'd expect you would be doing one or the other spec-wide. I'll do a subdir and we can change it if necessary | 21:14 |
dtroyer | s/wide/wise/ | 21:15 |
mtreinish | ok, sounds good | 21:15 |
*** david-lyle has joined #openstack-qa | 21:16 | |
*** yamamoto has quit IRC | 21:17 | |
*** mkollaro has quit IRC | 21:24 | |
*** Sukhdev has quit IRC | 21:24 | |
*** mpaolino has quit IRC | 21:25 | |
openstackgerrit | Merged openstack/tempest: Add scenario test for swift-proxy ceilometer middleware https://review.openstack.org/115971 | 21:30 |
openstackgerrit | Matt Riedemann proposed openstack-infra/elastic-recheck: Add query for testtools/tempest bug 1394747 https://review.openstack.org/136131 | 21:31 |
mriedem | mtreinish: jogo: ^ | 21:31 |
jogo | mriedem: glad that we fail on this now | 21:32 |
jogo | instead of passing silently | 21:32 |
mtreinish | jogo: it was pretty easy to fix | 21:34 |
*** weshay has joined #openstack-qa | 21:34 | |
mtreinish | sdague: it looks like tempest-lib from src still isn't quite working: http://logs.openstack.org/18/129418/3/check/gate-tempest-dsvm-src-tempest-lib/232e0b2/logs/devstacklog.txt.gz#_2014-11-20_03_55_13_991 | 21:36 |
openstackrecheck | openstack/requirements change: https://review.openstack.org/136018 failed gate-tempest-dsvm-full in the gate queue with an unrecognized error | 21:36 |
*** toure has quit IRC | 21:42 | |
mtreinish | mriedem, jogo: ^^^ looks like a swift test failure, don't see that very often | 21:42 |
mtreinish | oh, nm it's just a ceilo failure | 21:43 |
jogo | haha that makes more sense | 21:43 |
notmyname | lol | 21:43 |
mtreinish | jogo: I have a feeling that will be the new top flaky test. Another ceilo notification timeout | 21:45 |
jogo | mtreinish: are you working on the e-r query for it? | 21:45 |
mtreinish | I wasn't, but I can be | 21:46 |
jogo | mtreinish: cool | 21:47 |
*** melwitt has quit IRC | 21:47 | |
openstackgerrit | Merged openstack-infra/elastic-recheck: Add query for testtools/tempest bug 1394747 https://review.openstack.org/136131 | 21:49 |
*** e0ne has quit IRC | 21:50 | |
*** openstackrecheck has quit IRC | 21:50 | |
*** openstackrecheck has joined #openstack-qa | 21:50 | |
*** ajo has quit IRC | 21:55 | |
*** JayJ has quit IRC | 21:55 | |
mtreinish | jogo: well this log is really useful: http://logs.openstack.org/18/136018/1/gate/gate-tempest-dsvm-full/4c27858/logs/screen-ceilometer-api.txt.gz | 22:04 |
jogo | wow | 22:05 |
mtreinish | I think I'm just going to log on the console failure, I can't find anything else | 22:05 |
jogo | just wow | 22:05 |
jogo | notmyname: why can't your logs be this fun? | 22:05 |
*** jgrimm is now known as zz_jgrimm | 22:06 | |
mtreinish | jaypipes: ^^^ you're part of that logging wg thing right. Any thoughts on that one? | 22:07 |
*** ajo has joined #openstack-qa | 22:08 | |
jogo | clarkb: so the live migration stuff | 22:08 |
jogo | clarkb: tempest uses what nova reports back as the host | 22:08 |
*** melwitt has joined #openstack-qa | 22:09 | |
*** bkopilov has quit IRC | 22:09 | |
mtreinish | jogo: as opposed to it using a host nova doesn't return? :) | 22:10 |
jogo | as opposed to some IPs somehow | 22:11 |
jogo | mtreinish: it uses what nova returns from 'nova host-list' | 22:12 |
mtreinish | jogo: oh, ok :) I guess tempest could do a lookup or something, but that would seem weird | 22:12 |
jogo | which makes sense, so I think we should just update the host list | 22:12 |
jogo | clarkb: ^ | 22:12 |
*** rwsu has quit IRC | 22:12 | |
jogo | right | 22:12 |
clarkb | jogo: or we can set up name resolution | 22:12 |
clarkb | it won't be too hard | 22:12 |
lifeless | host files 4 eva | 22:13 |
*** markmcclain has quit IRC | 22:13 | |
jogo | lifeless: you need that on a knuckle tattoo | 22:14 |
jogo | clarkb: lets do name resolution then | 22:14 |
clarkb | jogo: basically do like we do for known hosts filling | 22:14 |
jogo | clarkb: and update the host file? | 22:14 |
clarkb | ask each node for its hostname in turn, put entry in master file for that node, copy resulting file everywhere | 22:14 |
clarkb | jogo: ya | 22:14 |
jogo | that is what I meant yeah | 22:14 |
lifeless | jogo: for this particular one yeah :) - you're traversing a well trodded path :) | 22:14 |
jogo | I thought you meant DNS | 22:14 |
clarkb | no dns is flaky in clouds | 22:15 |
jogo | clarkb: don't we build the clouds ;) | 22:15 |
*** rwsu_ has joined #openstack-qa | 22:15 | |
clarkb | jogo: and no one gets dns right | 22:15 |
clarkb | we should use WINS | 22:15 |
jogo | queue mordred rant | 22:16 |
clarkb | or netbios whatever that thing is called | 22:16 |
mordred | DNS | 22:16 |
mordred | it works | 22:16 |
clarkb | mordred: not in our clouds it doesn't | 22:16 |
mordred | learn the internet | 22:16 |
clarkb | its very saddening | 22:16 |
mordred | well | 22:16 |
mordred | that's because our clouds are crazy | 22:16 |
clarkb | mordred: did you see I added even more checks around that recently | 22:16 |
clarkb | it seems to have helped | 22:17 |
mordred | clarkb: sigh | 22:17 |
clarkb | as the name resolution stuff isn't high on e-r's list anymore | 22:17 |
clarkb | mordred: basically nodepool checks if the node can resolve git.o.o before making the node ready | 22:17 |
clarkb | if that fails then it deletes the node | 22:17 |
mordred | ++ | 22:17 |
jogo | clarkb: sounds like a great and sad hack | 22:17 |
jogo | do you know what the rate of failure is for that | 22:17 |
mordred | that's ridiculous, of course | 22:17 |
clarkb | jogo: e-r did | 22:17 |
*** rwsu_ has quit IRC | 22:19 | |
jogo | clarkb: for the hostname file | 22:22 |
sdague | mtreinish: yep, wrong call | 22:22 |
jogo | I should run hostname on each box? | 22:22 |
clarkb | jogo: ya you need a hosts file with a hostname to ip mapping of ever node | 22:22 |
clarkb | jogo: because you don't know which way migration will go | 22:22 |
clarkb | if VM is on primary node it will go to subnode and vice versa | 22:22 |
jogo | clarkb: or hostname -f | 22:23 |
clarkb | hrm its whatever nova uses | 22:23 |
clarkb | jogo: maybe use both to be safe? | 22:23 |
lifeless | mtreinish: sdague: any concerns about testtools-1.5.0 coming out after my lunch ? | 22:23 |
*** weshay has quit IRC | 22:23 | |
mtreinish | lifeless: not really, but if you could push up a d-g patch that installs it from source to test before the release that would be awesome | 22:24 |
jogo | clarkb: devstack-trusty-2-node-rax-dfw-3143753-382.slave.openstack.org | 22:24 |
openstackgerrit | Sean Dague proposed openstack-dev/devstack: fix tempest-lib setup https://review.openstack.org/136150 | 22:24 |
mtreinish | but I think we should be fine since the MRO fix landed | 22:24 |
sdague | mtreinish: ^^^ that should get you fixed | 22:25 |
clarkb | jogo: if that is what nova uses you will need the -f | 22:25 |
*** dustins has quit IRC | 22:25 | |
mtreinish | sdague: ok cool | 22:25 |
openstackgerrit | Matthew Treinish proposed openstack-infra/elastic-recheck: Add query for bug 1394774 https://review.openstack.org/136151 | 22:26 |
sdague | ok, dropping off for the day. Just rechecked all my backports because those got killed in the git storm. | 22:28 |
openstackgerrit | Matthew Treinish proposed openstack/tempest: Add check for too_slow_to_test flag on swift notification test https://review.openstack.org/136154 | 22:29 |
mtreinish | jogo: ^^^ I think that should fix the failure | 22:29 |
* mtreinish still thinks that's the best feature flag in tempest | 22:30 | |
lifeless | mtreinish: I think a bunch of upstream tests would be nice :) | 22:31 |
ianw | sdague: can you take a look at https://review.openstack.org/#/c/133914/ | 22:32 |
mtreinish | mriedem, jogo: https://review.openstack.org/136151 the only reason the hit count is so low is because icehouse is broken | 22:32 |
jogo | too slow to test lolg | 22:33 |
openstackgerrit | Dean Troyer proposed openstack/qa-specs: Add references to DevStack specs https://review.openstack.org/136156 | 22:33 |
mriedem | looking | 22:33 |
mriedem | mtreinish: we could pin that by branch and/or project probably | 22:35 |
*** gordc has joined #openstack-qa | 22:35 | |
mtreinish | mriedem: it'll hit every project but I can add a stable branch tag | 22:35 |
mriedem | ok | 22:35 |
gordc | mtreinish: thanks for opening bug... i added a comment to it... seems like a configuration issue which we may or may not want to address | 22:36 |
gordc | mtreinish: re: ceilometer swift bug | 22:36 |
mtreinish | gordc: yeah I'm thinking the flag check will be enough it's probably just on icehouse | 22:37 |
mtreinish | but we can leave it open for a bit after that tempest change lands and see if it clears up | 22:37 |
gordc | mtreinish: works for me. seems like a safer fix. | 22:37 |
openstackgerrit | Joe Gordon proposed openstack-infra/devstack-gate: DO NOT COMMIT disable config drive https://review.openstack.org/135831 | 22:37 |
openstackgerrit | Joe Gordon proposed openstack-infra/devstack-gate: DO NOT MERGE: testing aiopcpu with tempest full https://review.openstack.org/134639 | 22:37 |
openstackgerrit | Joe Gordon proposed openstack-infra/devstack-gate: Enable live block migration https://review.openstack.org/135768 | 22:37 |
openstackgerrit | Joe Gordon proposed openstack-infra/devstack-gate: Setup /etc/hosts for aiopcpu https://review.openstack.org/136158 | 22:37 |
*** MasterPiece has quit IRC | 22:37 | |
jogo | mtreinish clarkb: ok let see what happens | 22:37 |
*** andreykurilin_ has quit IRC | 22:38 | |
jogo | I have to go AFK for a bit so if you see a bug somewhere feel free to -1 and comment or fix it if you like | 22:38 |
*** dkranz has quit IRC | 22:38 | |
jogo | note I am expecting this to fail but with a different error now | 22:38 |
adam_g | jogo, managed to get setup with a multinode setup locally. playing around a bit trying to get ec2 metadata working, what error are chasing currently? | 22:38 |
openstackgerrit | Matthew Treinish proposed openstack-infra/elastic-recheck: Add query for bug 1394774 https://review.openstack.org/136151 | 22:38 |
mtreinish | mriedem: ^^^ | 22:38 |
mriedem | merged | 22:39 |
mtreinish | mriedem: cool, thanks | 22:39 |
gordc | mtreinish: i'm going to leave that bug open for ceilometer in case someone else wants to look at it and correct me tomorrow. | 22:40 |
*** mlavalle has quit IRC | 22:40 | |
mtreinish | mriedem: the reason it's only on requirements now is that icehouse jobs are broken everwhere else because of the taskflow reqs issue from this morning :) | 22:40 |
mtreinish | it's been my reqs fixes that have bounced off of it... | 22:40 |
mtreinish | gordc: ok cool, thanks | 22:40 |
openstackgerrit | Sergey Shnaidman proposed openstack/tempest: DHCPv6 network tests https://review.openstack.org/117458 | 22:41 |
openstackgerrit | Dean Troyer proposed openstack/qa-specs: Add references to DevStack specs https://review.openstack.org/136156 | 22:41 |
openstackgerrit | Merged openstack-infra/elastic-recheck: Add query for bug 1394774 https://review.openstack.org/136151 | 22:46 |
*** openstackrecheck has quit IRC | 22:51 | |
*** openstackrecheck has joined #openstack-qa | 22:51 | |
*** suneelb has quit IRC | 22:54 | |
*** suneelb has joined #openstack-qa | 22:55 | |
*** dims has quit IRC | 23:12 | |
*** ajo has quit IRC | 23:13 | |
*** yamamoto has joined #openstack-qa | 23:14 | |
openstackrecheck | openstack/requirements change: https://review.openstack.org/136018 failed because of: gate-tempest-dsvm-full: https://bugs.launchpad.net/bugs/1394774 | 23:15 |
*** yamamoto has quit IRC | 23:19 | |
openstackgerrit | Adam Gandelman proposed openstack-dev/devstack: Set metadata_host in nova.conf https://review.openstack.org/136176 | 23:20 |
*** mkollaro has joined #openstack-qa | 23:26 | |
*** dimtruck is now known as zz_dimtruck | 23:26 | |
openstackgerrit | Adam Gandelman proposed openstack-infra/devstack-gate: Enable nova-api-metadata for multi host nova-network https://review.openstack.org/136179 | 23:31 |
*** tkammer has quit IRC | 23:35 | |
*** gordc has quit IRC | 23:35 | |
*** toure has joined #openstack-qa | 23:41 | |
*** toure has quit IRC | 23:42 | |
*** mriedem has quit IRC | 23:53 | |
*** suneelb has quit IRC | 23:59 | |
*** suneelb has joined #openstack-qa | 23:59 | |
*** rbak has quit IRC | 23:59 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!