*** markvoelker has quit IRC | 00:00 | |
*** markvoelker has joined #openstack-nova | 00:01 | |
*** ijw has joined #openstack-nova | 00:05 | |
*** ijw has quit IRC | 00:10 | |
*** vishwanathj has joined #openstack-nova | 00:11 | |
mriedem | oh snap it merged | 00:15 |
---|---|---|
mriedem | dansmith: not yet no | 00:15 |
mriedem | the placement job is in the experimental queue in devstack btw | 00:15 |
mriedem | if you want to test that change | 00:15 |
openstackgerrit | OpenStack Proposal Bot proposed openstack/nova: Updated from global requirements https://review.openstack.org/363871 | 00:15 |
dansmith | yeah, but it should be good since we're healing ever couple minutes | 00:15 |
dansmith | not worried | 00:15 |
dansmith | merged -> woot | 00:15 |
*** ducttape_ has quit IRC | 00:17 | |
dansmith | mriedem: see my devstack change and explanation? | 00:17 |
mriedem | dansmith: yeah | 00:17 |
mriedem | i just checked it for the placement job | 00:17 |
mriedem | # NOTE(danms): LOL @ py3 | 00:18 |
mriedem | nice | 00:18 |
*** trisq has joined #openstack-nova | 00:19 | |
mriedem | ok i don't see anything else we need to wait for, there are 2 changes in the check queue for nova | 00:19 |
mriedem | i'm going to tag it and bag it | 00:19 |
dansmith | cool | 00:19 |
*** ducttape_ has joined #openstack-nova | 00:21 | |
mriedem | booya https://review.openstack.org/#/c/364600/ | 00:22 |
*** thorst has joined #openstack-nova | 00:24 | |
*** claudiub has quit IRC | 00:24 | |
*** ijw has joined #openstack-nova | 00:25 | |
*** ducttape_ has quit IRC | 00:25 | |
*** ijw has quit IRC | 00:26 | |
dansmith | still getting those vcpu warnings with 4x overcommit, not sure why | 00:28 |
dansmith | wonder if the allocation code isn't honoring those | 00:28 |
dansmith | hmm, seems like it is | 00:31 |
dansmith | oh well, I'll poke tomorrow | 00:31 |
*** annegentle has quit IRC | 00:31 | |
*** annegentle has joined #openstack-nova | 00:32 | |
*** anush__ has joined #openstack-nova | 00:34 | |
*** ijw has joined #openstack-nova | 00:35 | |
*** rfolco has quit IRC | 00:35 | |
*** markvoelker has quit IRC | 00:35 | |
*** praneshp has joined #openstack-nova | 00:36 | |
*** wanghao_ has joined #openstack-nova | 00:36 | |
*** tonytan4ever has joined #openstack-nova | 00:38 | |
*** ducttape_ has joined #openstack-nova | 00:39 | |
*** rfolco has joined #openstack-nova | 00:40 | |
*** rfolco has quit IRC | 00:40 | |
*** ducttape_ has quit IRC | 00:41 | |
*** diogogmt has joined #openstack-nova | 00:43 | |
openstackgerrit | liuxiuli proposed openstack/nova: Modify workflow of rebuild_instance when schedule fails https://review.openstack.org/348111 | 00:44 |
*** markvoelker has joined #openstack-nova | 00:44 | |
openstackgerrit | liuxiuli proposed openstack/nova: Modify request numa_topology when resize https://review.openstack.org/340270 | 00:44 |
openstackgerrit | zhufl proposed openstack/nova: Remove unused conf https://review.openstack.org/364031 | 00:45 |
*** zz_dimtruck is now known as dimtruck | 00:45 | |
*** 7F1AB46BF has joined #openstack-nova | 00:46 | |
*** Jeffrey4l has joined #openstack-nova | 00:46 | |
*** 7F1AB46BF has quit IRC | 00:48 | |
*** Swami__ has joined #openstack-nova | 00:48 | |
*** Hosam has quit IRC | 00:50 | |
*** kaisers_ has joined #openstack-nova | 00:51 | |
*** ljxiash has joined #openstack-nova | 00:51 | |
*** Swami has quit IRC | 00:52 | |
*** trisq has quit IRC | 00:55 | |
*** cheneydc has joined #openstack-nova | 00:55 | |
*** kaisers_ has quit IRC | 00:55 | |
*** ljxiash has quit IRC | 00:56 | |
*** ducttape_ has joined #openstack-nova | 00:59 | |
*** cheneydc has quit IRC | 01:00 | |
*** ducttape_ has quit IRC | 01:05 | |
*** ducttape_ has joined #openstack-nova | 01:09 | |
*** ijw has quit IRC | 01:10 | |
*** ijw has joined #openstack-nova | 01:11 | |
*** ijw has quit IRC | 01:11 | |
*** kfarr has joined #openstack-nova | 01:16 | |
*** salv-orlando has joined #openstack-nova | 01:18 | |
*** ijw has joined #openstack-nova | 01:20 | |
*** trisq has joined #openstack-nova | 01:20 | |
*** ijw has quit IRC | 01:24 | |
*** raunak has quit IRC | 01:24 | |
openstackgerrit | Takashi NATSUME proposed openstack/nova: Treat 'body_map' as a mapping of HTTP request bodies https://review.openstack.org/363457 | 01:25 |
*** salv-orlando has quit IRC | 01:27 | |
*** markvoelker has quit IRC | 01:28 | |
*** kfarr has quit IRC | 01:28 | |
*** akshai has joined #openstack-nova | 01:31 | |
*** ducttape_ has quit IRC | 01:31 | |
*** markvoelker has joined #openstack-nova | 01:35 | |
*** mtanino has quit IRC | 01:36 | |
*** annegentle has quit IRC | 01:37 | |
*** akshai has quit IRC | 01:37 | |
*** EinstCrazy has joined #openstack-nova | 01:38 | |
*** mriedem has quit IRC | 01:38 | |
*** mlavalle has quit IRC | 01:38 | |
*** wangqun_ has joined #openstack-nova | 01:40 | |
*** mriedem has joined #openstack-nova | 01:40 | |
*** markvoelker has quit IRC | 01:40 | |
*** ssurana has quit IRC | 01:44 | |
*** ducttape_ has joined #openstack-nova | 01:45 | |
*** gyee has quit IRC | 01:47 | |
*** ducttape_ has quit IRC | 01:47 | |
*** mriedem has quit IRC | 01:51 | |
*** ducttape_ has joined #openstack-nova | 01:52 | |
*** ljxiash has joined #openstack-nova | 01:56 | |
*** ljxiash_ has joined #openstack-nova | 01:58 | |
*** ljxiash__ has joined #openstack-nova | 02:01 | |
*** ljxiash has quit IRC | 02:01 | |
*** ljxiash_ has quit IRC | 02:02 | |
*** markvoelker has joined #openstack-nova | 02:04 | |
*** marst has quit IRC | 02:07 | |
*** marst has joined #openstack-nova | 02:07 | |
*** ducttape_ has quit IRC | 02:19 | |
*** browne has quit IRC | 02:19 | |
*** sdake has quit IRC | 02:21 | |
openstackgerrit | Eric Brown proposed openstack/nova: Update man pages with latest version and dates https://review.openstack.org/325441 | 02:21 |
*** unicell has quit IRC | 02:22 | |
*** su_zhang has joined #openstack-nova | 02:23 | |
*** markvoelker has quit IRC | 02:23 | |
*** Swami has joined #openstack-nova | 02:24 | |
*** tonytan4ever has quit IRC | 02:25 | |
*** Swami__ has quit IRC | 02:27 | |
*** hemanthm|bbl is now known as hemanthm | 02:28 | |
*** tonytan4ever has joined #openstack-nova | 02:29 | |
*** salv-orlando has joined #openstack-nova | 02:29 | |
*** Apoorva has quit IRC | 02:30 | |
*** salv-orlando has quit IRC | 02:32 | |
*** yamahata has quit IRC | 02:32 | |
*** Swami__ has joined #openstack-nova | 02:35 | |
*** su_zhang has quit IRC | 02:36 | |
*** su_zhang has joined #openstack-nova | 02:36 | |
*** Swami has quit IRC | 02:38 | |
*** Swami_ has quit IRC | 02:38 | |
*** Swami has joined #openstack-nova | 02:38 | |
*** kaisers_ has joined #openstack-nova | 02:40 | |
*** su_zhang has quit IRC | 02:40 | |
*** prateek_ has quit IRC | 02:43 | |
*** prateek_ has joined #openstack-nova | 02:43 | |
*** kaisers_ has quit IRC | 02:45 | |
*** thorst has quit IRC | 02:45 | |
*** takashin has quit IRC | 02:46 | |
*** thorst has joined #openstack-nova | 02:46 | |
*** ssurana has joined #openstack-nova | 02:48 | |
*** takashin has joined #openstack-nova | 02:49 | |
*** thorst has quit IRC | 02:54 | |
*** fragatina has quit IRC | 02:54 | |
*** dimtruck is now known as zz_dimtruck | 02:58 | |
*** rama_y has joined #openstack-nova | 03:09 | |
openstackgerrit | Eric Brown proposed openstack/nova: VMware: deprecate wsdl_location conf option https://review.openstack.org/342470 | 03:10 |
openstackgerrit | Eric Brown proposed openstack/nova: VMware: deprecate wsdl_location conf option https://review.openstack.org/342470 | 03:12 |
openstackgerrit | Bin Zhou proposed openstack/python-novaclient: Modify use of assertTrue(A in B) https://review.openstack.org/364637 | 03:13 |
*** mtanino has joined #openstack-nova | 03:16 | |
*** armax has quit IRC | 03:19 | |
*** raunak has joined #openstack-nova | 03:20 | |
*** anush__ has quit IRC | 03:33 | |
*** ducttape_ has joined #openstack-nova | 03:34 | |
*** ducttape_ has quit IRC | 03:36 | |
*** zz_dimtruck is now known as dimtruck | 03:37 | |
*** zerda2 has joined #openstack-nova | 03:38 | |
*** salv-orlando has joined #openstack-nova | 03:39 | |
*** diogogmt has quit IRC | 03:41 | |
*** coolsvap has joined #openstack-nova | 03:42 | |
*** intr1nsic has quit IRC | 03:43 | |
*** gouthamr_ has quit IRC | 03:45 | |
*** ianw has quit IRC | 03:46 | |
*** links has joined #openstack-nova | 03:46 | |
*** intr1nsic has joined #openstack-nova | 03:46 | |
*** tonytan_brb has joined #openstack-nova | 03:47 | |
*** salv-orlando has quit IRC | 03:47 | |
*** ijw has joined #openstack-nova | 03:47 | |
*** rfolco has joined #openstack-nova | 03:49 | |
*** tonytan4ever has quit IRC | 03:49 | |
*** ijw has quit IRC | 03:52 | |
*** thorst has joined #openstack-nova | 03:52 | |
*** thorst has quit IRC | 03:59 | |
openstackgerrit | Takashi NATSUME proposed openstack/nova: Enable cold migration with target host https://review.openstack.org/357105 | 04:02 |
*** dimtruck is now known as zz_dimtruck | 04:02 | |
*** ljxiash__ has quit IRC | 04:05 | |
*** ljxiash has joined #openstack-nova | 04:05 | |
*** ManishD has joined #openstack-nova | 04:09 | |
*** ljxiash has quit IRC | 04:10 | |
*** dikonoor has joined #openstack-nova | 04:16 | |
*** dikonoo has joined #openstack-nova | 04:16 | |
*** ManishD has quit IRC | 04:17 | |
*** ManishD has joined #openstack-nova | 04:17 | |
*** ljxiash has joined #openstack-nova | 04:19 | |
*** ianw has joined #openstack-nova | 04:21 | |
*** ljxiash has quit IRC | 04:24 | |
*** csmart has quit IRC | 04:35 | |
*** Swami__ has quit IRC | 04:35 | |
*** Swami has quit IRC | 04:35 | |
*** d34dh0r53 has quit IRC | 04:36 | |
*** ducttape_ has joined #openstack-nova | 04:37 | |
*** bhagyashri has quit IRC | 04:38 | |
*** rama_y has quit IRC | 04:39 | |
*** rfolco has quit IRC | 04:41 | |
*** ducttape_ has quit IRC | 04:42 | |
*** tonytan_brb has quit IRC | 04:44 | |
*** mtanino has quit IRC | 04:45 | |
*** d34dh0r53 has joined #openstack-nova | 04:47 | |
*** salv-orlando has joined #openstack-nova | 04:50 | |
*** prateek_ has quit IRC | 04:52 | |
*** gongysh has quit IRC | 04:54 | |
*** thorst has joined #openstack-nova | 04:56 | |
*** prateek has joined #openstack-nova | 04:58 | |
*** salv-orlando has quit IRC | 04:59 | |
*** hparekh has joined #openstack-nova | 05:02 | |
*** prateek has quit IRC | 05:04 | |
*** thorst has quit IRC | 05:04 | |
*** hparekh has left #openstack-nova | 05:07 | |
*** fungi has quit IRC | 05:08 | |
*** lbragstad has quit IRC | 05:09 | |
*** raunak has quit IRC | 05:09 | |
-openstackstatus- NOTICE: The logs volume is full causing jobs to fail with POST_FAILURE. This is being worked on, please do not recheck until notified. | 05:09 | |
*** ChanServ changes topic to "The logs volume is full causing jobs to fail with POST_FAILURE. This is being worked on, please do not recheck until notified." | 05:09 | |
*** tr3buchet has quit IRC | 05:10 | |
*** gongysh has joined #openstack-nova | 05:10 | |
*** nortman has quit IRC | 05:10 | |
*** belliott has quit IRC | 05:10 | |
*** claudiub has joined #openstack-nova | 05:11 | |
*** ecelik has joined #openstack-nova | 05:17 | |
*** lbragstad has joined #openstack-nova | 05:18 | |
*** unicell has joined #openstack-nova | 05:20 | |
*** prateek has joined #openstack-nova | 05:20 | |
*** coolsvap is now known as coolsvap_ | 05:21 | |
*** mamandle has joined #openstack-nova | 05:21 | |
*** fungi has joined #openstack-nova | 05:22 | |
*** tr3buchet has joined #openstack-nova | 05:23 | |
*** nortman has joined #openstack-nova | 05:24 | |
*** moshele has joined #openstack-nova | 05:26 | |
*** ssurana has quit IRC | 05:26 | |
*** takashin has left #openstack-nova | 05:29 | |
*** ChanServ changes topic to "This channel is for Nova development. For support of Nova deployments, please use #openstack. Please see: https://wiki.openstack.org/wiki/Nova/Newton_Release_Schedule" | 05:35 | |
-openstackstatus- NOTICE: Space has been freed up on the log server. If you have POST_FAILURE results it is now safe to issue a 'recheck' | 05:36 | |
*** moshele has quit IRC | 05:37 | |
*** ssurana has joined #openstack-nova | 05:37 | |
*** ducttape_ has joined #openstack-nova | 05:38 | |
*** moshele has joined #openstack-nova | 05:42 | |
*** ducttape_ has quit IRC | 05:42 | |
*** belliott has joined #openstack-nova | 05:43 | |
*** mjura has joined #openstack-nova | 05:49 | |
*** ljxiash has joined #openstack-nova | 05:52 | |
*** lpetrut has joined #openstack-nova | 05:55 | |
*** mjura has quit IRC | 05:56 | |
*** mjura has joined #openstack-nova | 05:56 | |
*** nkrinner_afk is now known as nkrinner | 05:57 | |
*** ssurana has quit IRC | 06:00 | |
*** csmart_ has joined #openstack-nova | 06:00 | |
*** csmart_ is now known as csmart | 06:02 | |
*** thorst has joined #openstack-nova | 06:02 | |
*** ttrumm has joined #openstack-nova | 06:02 | |
*** bhagyashri has joined #openstack-nova | 06:02 | |
*** paul-carlton2 has joined #openstack-nova | 06:08 | |
*** artom has quit IRC | 06:08 | |
*** thorst has quit IRC | 06:09 | |
*** salv-orlando has joined #openstack-nova | 06:16 | |
*** moshele has quit IRC | 06:17 | |
*** kaisers_ has joined #openstack-nova | 06:17 | |
*** artom has joined #openstack-nova | 06:20 | |
*** kaisers_ has quit IRC | 06:23 | |
*** ljxiash has quit IRC | 06:23 | |
*** praneshp has quit IRC | 06:25 | |
*** unicell1 has joined #openstack-nova | 06:25 | |
*** unicell has quit IRC | 06:25 | |
openstackgerrit | Slawek Kaplonski proposed openstack/nova: Fix race condition bug during live_snapshot https://review.openstack.org/363926 | 06:27 |
*** woodster_ has quit IRC | 06:29 | |
*** thorst has joined #openstack-nova | 06:36 | |
*** ducttape_ has joined #openstack-nova | 06:39 | |
*** rcernin has joined #openstack-nova | 06:40 | |
*** ducttape_ has quit IRC | 06:44 | |
*** thorst has quit IRC | 06:44 | |
*** vishwanathj is now known as vishwanathj_zzz | 06:45 | |
openstackgerrit | Hussain Chachuliya proposed openstack/nova: Refactor: Move post method to APIValidationTestCase base class https://review.openstack.org/364687 | 06:45 |
*** andreas_s has joined #openstack-nova | 06:48 | |
*** pcaruana has joined #openstack-nova | 06:48 | |
*** gongysh has quit IRC | 06:51 | |
*** bsv has joined #openstack-nova | 06:52 | |
*** vanadrig has joined #openstack-nova | 06:54 | |
*** tesseract- has joined #openstack-nova | 07:02 | |
openstackgerrit | Tomonobu Niwa proposed openstack/nova: incorrect description in nova-api.log about quota check https://review.openstack.org/364698 | 07:06 |
*** sahid has joined #openstack-nova | 07:07 | |
*** brad[] has quit IRC | 07:09 | |
*** brad[] has joined #openstack-nova | 07:10 | |
*** salv-orlando has quit IRC | 07:14 | |
*** kaisers_ has joined #openstack-nova | 07:15 | |
*** mpaolino has joined #openstack-nova | 07:16 | |
*** rmart04 has joined #openstack-nova | 07:17 | |
openstackgerrit | Hussain Chachuliya proposed openstack/nova: Refactor: Move post method to APIValidationTestCase base class https://review.openstack.org/364687 | 07:19 |
*** lpetrut has quit IRC | 07:27 | |
*** hussain_c has joined #openstack-nova | 07:29 | |
openstackgerrit | Balazs Gibizer proposed openstack/nova: Removed enum duplication from nova.compute https://review.openstack.org/335066 | 07:30 |
*** sridharg has joined #openstack-nova | 07:31 | |
*** Oku_OS-away is now known as Oku_OS | 07:33 | |
*** markus_z has joined #openstack-nova | 07:33 | |
*** kaisers_ has quit IRC | 07:34 | |
*** matrohon has joined #openstack-nova | 07:42 | |
*** lpetrut has joined #openstack-nova | 07:42 | |
*** thorst has joined #openstack-nova | 07:42 | |
hussain_c | jaypipes, alaski: Hi, could you please take a look at this small code optimization patch? https://review.openstack.org/#/c/364687/ | 07:43 |
*** amotoki has joined #openstack-nova | 07:44 | |
*** oanson has joined #openstack-nova | 07:46 | |
*** lpetrut has quit IRC | 07:46 | |
*** thorst has quit IRC | 07:48 | |
*** ducttape_ has joined #openstack-nova | 07:49 | |
*** sameo has joined #openstack-nova | 07:51 | |
*** ducttape_ has quit IRC | 07:53 | |
*** ttrumm has quit IRC | 07:54 | |
*** ttrumm has joined #openstack-nova | 07:55 | |
*** bsv has quit IRC | 07:58 | |
*** zzzeek has quit IRC | 08:00 | |
*** paul-carlton2 has quit IRC | 08:00 | |
*** zzzeek has joined #openstack-nova | 08:00 | |
*** oanson has quit IRC | 08:00 | |
openstackgerrit | Tomonobu Niwa proposed openstack/nova: incorrect description in nova-api.log about quota check https://review.openstack.org/364698 | 08:01 |
openstackgerrit | JiWei proposed openstack/nova: 'unmap' is written without considering the configuration https://review.openstack.org/364122 | 08:01 |
*** gszasz has joined #openstack-nova | 08:02 | |
*** yassine has joined #openstack-nova | 08:03 | |
*** yassine is now known as Guest55526 | 08:03 | |
*** ttrumm has quit IRC | 08:04 | |
*** ttrumm has joined #openstack-nova | 08:04 | |
*** lucas-dinner is now known as lucasagomes | 08:05 | |
*** derekh has joined #openstack-nova | 08:08 | |
*** bauzas is now known as bauwser | 08:10 | |
*** xinliang has quit IRC | 08:11 | |
*** kaisers_ has joined #openstack-nova | 08:12 | |
*** mamandle has quit IRC | 08:12 | |
*** mamandle has joined #openstack-nova | 08:17 | |
openstackgerrit | yuanyue proposed openstack/nova: Fix incorrect debug information about quota check https://review.openstack.org/364752 | 08:21 |
*** gongysh has joined #openstack-nova | 08:23 | |
*** xinliang has joined #openstack-nova | 08:24 | |
*** salv-orlando has joined #openstack-nova | 08:25 | |
*** paul-carlton2 has joined #openstack-nova | 08:28 | |
*** mamandle_ has joined #openstack-nova | 08:29 | |
*** mamandle has quit IRC | 08:31 | |
*** mamandle_ is now known as mamandle | 08:31 | |
*** cdent has joined #openstack-nova | 08:34 | |
*** oanson has joined #openstack-nova | 08:35 | |
openstackgerrit | Claudiu Belu proposed openstack/nova: numa: Fixes NUMA topology related unit tests https://review.openstack.org/278835 | 08:42 |
openstackgerrit | Claudiu Belu proposed openstack/nova: Fixes python 3 unit tests https://review.openstack.org/278810 | 08:42 |
openstackgerrit | Claudiu Belu proposed openstack/nova: compute: fixes python 3 related unit tests https://review.openstack.org/300262 | 08:42 |
openstackgerrit | Chris Dent proposed openstack/nova: Optional separate database for placement API https://review.openstack.org/362766 | 08:44 |
*** snikitin_ has joined #openstack-nova | 08:44 | |
*** thorst has joined #openstack-nova | 08:46 | |
*** rmart04 has quit IRC | 08:47 | |
*** gongysh has quit IRC | 08:48 | |
*** ducttape_ has joined #openstack-nova | 08:50 | |
*** rmart04 has joined #openstack-nova | 08:54 | |
*** thorst has quit IRC | 08:54 | |
*** mgoddard has quit IRC | 08:54 | |
*** ducttape_ has quit IRC | 08:54 | |
*** vanadrig has quit IRC | 08:56 | |
snikitin_ | Hi folks! I have a one question. Is it possible to backport bug fix which adds some new fields to nova object. This fix doesn't change API or rpc. just nova object. https://review.openstack.org/#/c/307124/21/nova/objects/migration_context.py | 08:56 |
snikitin_ | ? | 08:57 |
*** sudipto has joined #openstack-nova | 08:58 | |
*** sudipto_ has joined #openstack-nova | 08:58 | |
cdent | snikitin_: I'm not expert on that stuff, but adding fields would require a version change on the object which I guess make it undesirable as a backport | 08:59 |
openstackgerrit | Chris Dent proposed openstack/nova: Add {get_,set_}aggregates to objects.ResourceProvider https://review.openstack.org/355263 | 08:59 |
openstackgerrit | Chris Dent proposed openstack/nova: Implement get and set aggregates in the placement API https://review.openstack.org/357993 | 08:59 |
openstackgerrit | Chris Dent proposed openstack/nova: [placement] add a placement_aggregates table to api_db https://review.openstack.org/362863 | 08:59 |
*** mgoddard has joined #openstack-nova | 09:03 | |
prateek | johnthetubaguy, can you please review https://review.openstack.org/#/c/294513/ when you have some free time ? | 09:03 |
*** EinstCra_ has joined #openstack-nova | 09:03 | |
*** snikitin_ has quit IRC | 09:04 | |
*** code-R has joined #openstack-nova | 09:06 | |
*** EinstCrazy has quit IRC | 09:06 | |
*** jaosorior has joined #openstack-nova | 09:11 | |
*** code-R_ has joined #openstack-nova | 09:11 | |
*** zenoway has joined #openstack-nova | 09:13 | |
*** code-R has quit IRC | 09:13 | |
*** lpetrut has joined #openstack-nova | 09:16 | |
*** jaypipes has quit IRC | 09:20 | |
*** bks has joined #openstack-nova | 09:20 | |
*** kumarb has joined #openstack-nova | 09:27 | |
*** bks has quit IRC | 09:30 | |
*** kumarb has quit IRC | 09:31 | |
*** bks has joined #openstack-nova | 09:32 | |
*** mamandle has quit IRC | 09:36 | |
*** sudipto has quit IRC | 09:40 | |
*** sudipto_ has quit IRC | 09:40 | |
openstackgerrit | Stephen Finucane proposed openstack/nova-specs: Reserve NUMA nodes with PCI devices attached https://review.openstack.org/364468 | 09:41 |
*** EinstCra_ has quit IRC | 09:48 | |
*** bks has quit IRC | 09:48 | |
*** bks has joined #openstack-nova | 09:50 | |
*** ducttape_ has joined #openstack-nova | 09:50 | |
*** thorst has joined #openstack-nova | 09:52 | |
*** ducttape_ has quit IRC | 09:55 | |
*** bhagyashri has quit IRC | 09:58 | |
*** thorst has quit IRC | 09:58 | |
openstackgerrit | OpenStack Proposal Bot proposed openstack/nova: Updated from global requirements https://review.openstack.org/363871 | 09:59 |
*** EinstCrazy has joined #openstack-nova | 10:02 | |
*** liuyulong is now known as yulong|away | 10:02 | |
*** EinstCrazy has quit IRC | 10:05 | |
*** snikitin_ has joined #openstack-nova | 10:09 | |
openstackgerrit | Davanum Srinivas (dims) proposed openstack/nova: [WIP] Testing latest u-c https://review.openstack.org/303345 | 10:10 |
*** markvoelker has joined #openstack-nova | 10:10 | |
*** EinstCrazy has joined #openstack-nova | 10:10 | |
*** rtheis has joined #openstack-nova | 10:11 | |
*** EinstCrazy has quit IRC | 10:13 | |
*** EinstCrazy has joined #openstack-nova | 10:13 | |
*** johnthetubaguy_w has joined #openstack-nova | 10:13 | |
*** EinstCrazy has quit IRC | 10:14 | |
*** rmart04 has quit IRC | 10:14 | |
*** EinstCrazy has joined #openstack-nova | 10:15 | |
openstackgerrit | Krzysztof ÅšwiÄ…tek proposed openstack/nova: Fix issue with not removing rbd rescue disk https://review.openstack.org/314928 | 10:15 |
*** kevinbenton has quit IRC | 10:15 | |
*** kevinbenton has joined #openstack-nova | 10:17 | |
*** wangqun_ has quit IRC | 10:17 | |
kswiatek | sdague, pkoniszewski, melwitt, mdbooth, could you review: https://review.openstack.org/#/c/314928/ | 10:18 |
*** rmart04 has joined #openstack-nova | 10:19 | |
kswiatek | and I have question, I heard if I add specfic tags (related to review), it will speed up a little bit searching this review, and so reviewing process. If it's true what tags I can add? :) | 10:20 |
*** markvoelker has quit IRC | 10:22 | |
*** johnthetubaguy_w has quit IRC | 10:23 | |
openstackgerrit | Stephen Finucane proposed openstack/nova: Forcing a configdrive in cfg should populate DB https://review.openstack.org/364814 | 10:23 |
*** EinstCrazy has quit IRC | 10:24 | |
*** tbachman has quit IRC | 10:27 | |
*** markvoelker has joined #openstack-nova | 10:30 | |
mdbooth | kashyap: https://bugs.launchpad.net/nova/+bug/1619606 | 10:31 |
openstack | Launchpad bug 1619606 in OpenStack Compute (nova) "snapshot_volume_backed races, could result in data corruption" [Undecided,New] | 10:31 |
* kashyap clicks | 10:32 | |
kashyap | mdbooth: Speaking of races, another fun "race" -- https://review.openstack.org/#/c/363926/3 "Fix race condition bug during live_snapshot" | 10:32 |
*** mpaolino has quit IRC | 10:33 | |
mdbooth | kashyap: Fun! | 10:34 |
kashyap | I did a write-up to libvirt upstream, and they agreed it's a valid bug | 10:34 |
openstackgerrit | Gábor Antal proposed openstack/nova: Throw exception if hw:numa_nodes is set to 0 https://review.openstack.org/337380 | 10:34 |
mdbooth | kashyap: I don't understand how that patch fixes it, though | 10:34 |
kashyap | See Eric's (as usual, eloquent) response to my question and context -- http://www.redhat.com/archives/libvir-list/2016-September/msg00011.html | 10:34 |
mdbooth | If libvirt returns 0, then presumably 'cur/end' are still in the returned status, but with values of 0 | 10:35 |
mdbooth | So setting a default of -1 isn't going to be effective, right? | 10:35 |
*** trisq has quit IRC | 10:35 | |
kashyap | mdbooth: Setting a default of "-1" means | 10:35 |
kashyap | Don't complete the job yet | 10:36 |
* mdbooth should look at the caller | 10:36 | |
kashyap | I.e. to finish the blockRebase() operation | 10:36 |
kashyap | Libvirt issues blockJobAbort(), too soon | 10:36 |
*** zenoway has quit IRC | 10:36 | |
kashyap | Here, "too soon" == before QEMU reports the status as "ready": true | 10:37 |
*** zenoway has joined #openstack-nova | 10:37 | |
kashyap | If you have time, this walkthrough of the libvirt <-> QEMU traffic clearly demonstrates it -- https://bugs.launchpad.net/nova/+bug/1530275/comments/16 | 10:37 |
openstack | Launchpad bug 1530275 in OpenStack Compute (nova) "Live snapshot is corrupted (possibly race condition?)" [Undecided,In progress] - Assigned to Slawek Kaplonski (slaweq) | 10:37 |
* mdbooth notes that wait_for_job doesn't do any waiting :/ | 10:37 | |
*** markvoelker has quit IRC | 10:40 | |
mdbooth | Ah... there are 2 methods called get_job_info | 10:40 |
kashyap | mdbooth: Yeah, too many wrappers :-( | 10:41 |
*** zenoway has quit IRC | 10:41 | |
openstackgerrit | Lucian Petrut proposed openstack/nova: HyperV: ensure config drives are copied as well during resizes https://review.openstack.org/364829 | 10:43 |
mdbooth | kashyap: So if the job hasn't started yet, end ==0, and cur is not set? | 10:43 |
kashyap | mdbooth: No: | 10:44 |
kashyap | mdbooth: "If qemu doesn't report anything (because the job is not started yet), then libvirt should report cur=0, end=1 (the job still has 100% to go)." | 10:44 |
*** xinliang has quit IRC | 10:44 | |
kashyap | There are 6 possible scenarios here. Eric outlines potential way to address them in the above thread. | 10:45 |
* mdbooth reads eric's mail | 10:45 | |
openstackgerrit | Slawek Kaplonski proposed openstack/nova: Fix race condition bug during live_snapshot https://review.openstack.org/363926 | 10:50 |
*** zenoway has joined #openstack-nova | 10:50 | |
*** Dinesh_Bhor has quit IRC | 10:51 | |
*** sahid has quit IRC | 10:51 | |
*** ducttape_ has joined #openstack-nova | 10:51 | |
*** Dinesh_Bhor has joined #openstack-nova | 10:51 | |
*** paul-carlton2 has quit IRC | 10:52 | |
kashyap | mdbooth: While reading, Eric made a little typo there, wherever you see "done" in quotes, replace it with "ready", that's what QEMU calls that string as. | 10:54 |
mdbooth | kashyap: Can you think of a case where end might legitimately be zero? | 10:55 |
mdbooth | A zero-sized block device? | 10:55 |
*** thorst has joined #openstack-nova | 10:56 | |
kashyap | mdbooth: Yeah, that could possibly one case. But what would one do with a zero-size block device? (Stashed it to populate it with contents later?) | 10:56 |
*** ducttape_ has quit IRC | 10:56 | |
mdbooth | Eric describes a future where libvirt returns cur != end, but right now cur=0, end=0 is theoretically ambiguous, right? | 10:56 |
* kashyap thinks a bit more... | 10:56 | |
mdbooth | If end was really 0, and this signalled completion, then we'd hang forever | 10:57 |
kashyap | mdbooth: Yes, indeed it is ambigious, as it stands. | 10:57 |
mdbooth | But we could potentially work round that elsewhere | 10:57 |
*** xinliang has joined #openstack-nova | 10:57 | |
*** zul has quit IRC | 10:58 | |
kashyap | If the job hasn't started, QEMU reports cur == end == 0, which can make Nova think: "Oh, the job seems to be completed, let's abort it" | 10:58 |
*** slaweq has quit IRC | 10:58 | |
* kashyap 's brain hurts thinking of all these cases. | 10:58 | |
*** paul-carlton2 has joined #openstack-nova | 10:58 | |
*** zenoway has quit IRC | 10:58 | |
*** zenoway has joined #openstack-nova | 11:01 | |
*** roeyc has joined #openstack-nova | 11:01 | |
*** thorst has quit IRC | 11:04 | |
*** abalutoiu has joined #openstack-nova | 11:06 | |
johnthetubaguy | raj_singh: do you know if anyway is looking at updating the compute conf reference dock thing? | 11:06 |
*** gszasz has quit IRC | 11:07 | |
*** prateek has quit IRC | 11:07 | |
*** claudiub has quit IRC | 11:07 | |
*** dikonoor has quit IRC | 11:07 | |
*** dikonoo has quit IRC | 11:07 | |
*** cdelatte has quit IRC | 11:07 | |
*** jgrassler has quit IRC | 11:07 | |
*** dulek has quit IRC | 11:07 | |
*** lennyb has quit IRC | 11:07 | |
*** cfriesen has quit IRC | 11:07 | |
*** kswiatek has quit IRC | 11:07 | |
*** darrenc has quit IRC | 11:07 | |
*** tpatzig_ has quit IRC | 11:07 | |
*** gstepanov__ has quit IRC | 11:07 | |
*** lamt has quit IRC | 11:07 | |
*** raj_singh has quit IRC | 11:07 | |
*** anteaya has quit IRC | 11:07 | |
*** niedbalski has quit IRC | 11:07 | |
*** eliqiao has quit IRC | 11:07 | |
*** ejat has quit IRC | 11:07 | |
openstackgerrit | liuxiuli proposed openstack/nova: Modify request numa_topology when resize https://review.openstack.org/340270 | 11:08 |
johnthetubaguy | Hmm, bummer, net splitting again | 11:08 |
*** dikonoor has joined #openstack-nova | 11:09 | |
*** dikonoo has joined #openstack-nova | 11:09 | |
*** gszasz has joined #openstack-nova | 11:09 | |
*** prateek has joined #openstack-nova | 11:09 | |
*** claudiub has joined #openstack-nova | 11:09 | |
*** cdelatte has joined #openstack-nova | 11:09 | |
*** dulek has joined #openstack-nova | 11:09 | |
*** lennyb has joined #openstack-nova | 11:09 | |
*** cfriesen has joined #openstack-nova | 11:09 | |
*** kswiatek has joined #openstack-nova | 11:09 | |
*** darrenc has joined #openstack-nova | 11:09 | |
*** tpatzig_ has joined #openstack-nova | 11:09 | |
*** gstepanov__ has joined #openstack-nova | 11:09 | |
*** lamt has joined #openstack-nova | 11:09 | |
*** raj_singh has joined #openstack-nova | 11:09 | |
*** anteaya has joined #openstack-nova | 11:09 | |
*** niedbalski has joined #openstack-nova | 11:09 | |
*** eliqiao has joined #openstack-nova | 11:09 | |
*** ejat has joined #openstack-nova | 11:09 | |
*** mamandle has joined #openstack-nova | 11:12 | |
*** roeyc has quit IRC | 11:14 | |
*** jgrassler has joined #openstack-nova | 11:14 | |
*** cdelatte has quit IRC | 11:15 | |
*** dikonoor has quit IRC | 11:24 | |
*** dikonoo has quit IRC | 11:25 | |
*** dikonoor has joined #openstack-nova | 11:25 | |
*** dikonoo has joined #openstack-nova | 11:25 | |
*** bvanhav has joined #openstack-nova | 11:26 | |
*** cdelatte has joined #openstack-nova | 11:30 | |
*** zul has joined #openstack-nova | 11:30 | |
*** trisq has joined #openstack-nova | 11:37 | |
*** baoli has joined #openstack-nova | 11:39 | |
*** lucasagomes is now known as lucas-hungry | 11:40 | |
*** aloga has quit IRC | 11:41 | |
*** bks has quit IRC | 11:42 | |
cdent | p!spy 2 | 11:44 |
*** Fdaisuke has quit IRC | 11:46 | |
*** gouthamr has joined #openstack-nova | 11:49 | |
*** ducttape_ has joined #openstack-nova | 11:52 | |
*** prateek has quit IRC | 11:53 | |
*** xinliang has quit IRC | 11:54 | |
*** baoli_ has joined #openstack-nova | 11:54 | |
*** ducttape_ has quit IRC | 11:56 | |
*** thorst has joined #openstack-nova | 11:57 | |
*** baoli has quit IRC | 11:57 | |
*** tbachman has joined #openstack-nova | 12:01 | |
*** tblakes has joined #openstack-nova | 12:01 | |
*** derekh has quit IRC | 12:02 | |
*** tblakes_ has joined #openstack-nova | 12:02 | |
csatari | I would like to ask for help with api_sample_tests. | 12:03 |
csatari | I woudld like to test a new microversion, let say 2.39 | 12:03 |
*** pilgrimstack has quit IRC | 12:03 | |
*** gongysh has joined #openstack-nova | 12:03 | |
csatari | I've created a v2.39 directory under .../servers | 12:04 |
csatari | and added the relevant .json.tpl files. | 12:04 |
csatari | When I run the testcase I get the following error: | 12:05 |
csatari | No such file or directory: '/run/media/csatari/MyVolume/work/OpenStack/nova/nova/doc/api_samples/servers/v2.39/servers-list-resp.json | 12:05 |
*** ekarlso_ has joined #openstack-nova | 12:05 | |
*** pilgrimstack has joined #openstack-nova | 12:05 | |
csatari | However I have a servers-list.resp.tpl there. | 12:06 |
*** avolkov has joined #openstack-nova | 12:06 | |
*** avolkov has left #openstack-nova | 12:06 | |
*** tblakes has quit IRC | 12:06 | |
*** tblakes_ is now known as tblakes | 12:07 | |
*** baoli has joined #openstack-nova | 12:08 | |
*** Hosam has joined #openstack-nova | 12:09 | |
*** tbachman_ has joined #openstack-nova | 12:09 | |
*** tbachman has quit IRC | 12:10 | |
*** tbachman_ is now known as tbachman | 12:10 | |
*** baoli__ has joined #openstack-nova | 12:10 | |
*** ekarlso_ has quit IRC | 12:11 | |
*** salv-orlando has quit IRC | 12:11 | |
*** flaper87 has quit IRC | 12:11 | |
openstackgerrit | Breno Leitão proposed openstack/nova: libvirt: add hugepages support for Power https://review.openstack.org/303564 | 12:11 |
*** baoli_ has quit IRC | 12:12 | |
*** baoli has quit IRC | 12:13 | |
*** ducttape_ has joined #openstack-nova | 12:14 | |
*** zul_ has joined #openstack-nova | 12:15 | |
*** zul_ has quit IRC | 12:16 | |
*** zul_ has joined #openstack-nova | 12:17 | |
*** zengine has joined #openstack-nova | 12:17 | |
*** zul has quit IRC | 12:17 | |
*** zengine has quit IRC | 12:20 | |
*** mamandle has quit IRC | 12:24 | |
*** sudipto_ has joined #openstack-nova | 12:24 | |
*** sudipto has joined #openstack-nova | 12:24 | |
*** andreas_s has quit IRC | 12:26 | |
*** rodrigods has quit IRC | 12:27 | |
*** rodrigods has joined #openstack-nova | 12:27 | |
gibi | syjulian: hi! finally I managed to write up the json schema bp for nova for ocata https://blueprints.launchpad.net/nova/+spec/json-schema-for-versioned-notifications | 12:29 |
*** flaper87 has joined #openstack-nova | 12:29 | |
*** flaper87 has quit IRC | 12:29 | |
*** flaper87 has joined #openstack-nova | 12:29 | |
*** sahid has joined #openstack-nova | 12:30 | |
*** rfolco has joined #openstack-nova | 12:30 | |
*** trisq has quit IRC | 12:35 | |
*** xinliang has joined #openstack-nova | 12:38 | |
*** annegentle has joined #openstack-nova | 12:38 | |
*** lucas-hungry is now known as lucasagomes | 12:38 | |
*** vladikr has joined #openstack-nova | 12:41 | |
*** rlrossit has joined #openstack-nova | 12:42 | |
*** zul_ has quit IRC | 12:42 | |
*** jaosorior has quit IRC | 12:43 | |
*** jaosorior has joined #openstack-nova | 12:43 | |
*** zul has joined #openstack-nova | 12:43 | |
*** zul has quit IRC | 12:45 | |
*** zul has joined #openstack-nova | 12:45 | |
*** tbachman has quit IRC | 12:45 | |
*** trisq has joined #openstack-nova | 12:45 | |
*** prateek has joined #openstack-nova | 12:47 | |
*** alaski is now known as lascii | 12:49 | |
*** mnestratov has joined #openstack-nova | 12:49 | |
*** PaulMurray has joined #openstack-nova | 12:51 | |
*** gouthamr_ has joined #openstack-nova | 12:51 | |
*** edleafe is now known as figleaf | 12:51 | |
*** ducttape_ has quit IRC | 12:52 | |
*** dikonoor has quit IRC | 12:52 | |
*** annegentle has quit IRC | 12:52 | |
*** gouthamr has quit IRC | 12:55 | |
*** woodster_ has joined #openstack-nova | 12:56 | |
*** erhudy has joined #openstack-nova | 12:58 | |
*** gszasz has quit IRC | 13:00 | |
*** andymaier has joined #openstack-nova | 13:00 | |
*** syjulian_ has joined #openstack-nova | 13:04 | |
*** ftersin has joined #openstack-nova | 13:04 | |
*** trisq has quit IRC | 13:06 | |
*** salv-orlando has joined #openstack-nova | 13:06 | |
*** ManishD has quit IRC | 13:08 | |
*** xyang1 has joined #openstack-nova | 13:08 | |
*** jaosorior has quit IRC | 13:09 | |
*** tongli has joined #openstack-nova | 13:09 | |
*** ttrumm has quit IRC | 13:10 | |
*** ttrumm has joined #openstack-nova | 13:11 | |
*** ttrumm has quit IRC | 13:12 | |
*** TravT has joined #openstack-nova | 13:13 | |
*** BjoernT has joined #openstack-nova | 13:14 | |
*** BjoernT is now known as Bjoern_zZzZzZzZ | 13:14 | |
*** aloga has joined #openstack-nova | 13:14 | |
*** tbachman has joined #openstack-nova | 13:16 | |
*** mriedem has joined #openstack-nova | 13:16 | |
mriedem | this is holding up another change on stable/mitaka, can someone take a look at this backport? https://review.openstack.org/#/c/357566/ | 13:17 |
bauwser | mriedem: on it, but only +1ing :) | 13:19 |
mriedem | it's the thought that counts | 13:19 |
bauwser | the 3rd-party CI is green and the change looks reasonable, so I don't expect too much of debate here :) | 13:20 |
openstackgerrit | Alberto Planas proposed openstack/nova: Add migrate_uri for invoking the migration https://review.openstack.org/356558 | 13:22 |
*** Bjoern_zZzZzZzZ is now known as BjoernT | 13:22 | |
*** eharney has joined #openstack-nova | 13:22 | |
mdbooth | mriedem bauwser: Incidentally, I filed this bug earlier today: https://bugs.launchpad.net/nova/+bug/1619606 | 13:23 |
openstack | Launchpad bug 1619606 in OpenStack Compute (nova) "snapshot_volume_backed races, could result in data corruption" [Undecided,New] | 13:23 |
*** mmedvede has quit IRC | 13:23 | |
mdbooth | It might interact with this. | 13:23 |
mriedem | well, if we set a task state in the api then the 2nd volume-backed snapshot shouldn't get in right? | 13:25 |
mdbooth | mriedem: Right. | 13:26 |
mdbooth | I think it's a 'simple' fix | 13:26 |
*** ducttape_ has joined #openstack-nova | 13:26 | |
*** leifz has quit IRC | 13:26 | |
*** rlrossit has quit IRC | 13:26 | |
*** zz_dimtruck has quit IRC | 13:26 | |
mdbooth | In a, it-sounds-simple-but-the-potential-interactions-are-really-scary kind of way | 13:26 |
*** wendar has quit IRC | 13:27 | |
*** ayoung has joined #openstack-nova | 13:28 | |
*** rlrossit has joined #openstack-nova | 13:29 | |
mdbooth | mriedem: Unless you were asking if snapshot_volume_backed() is called by snapshot(), and it's all therefore ok? | 13:29 |
mdbooth | If so, weirdly not. | 13:29 |
*** wendar has joined #openstack-nova | 13:29 | |
mriedem | nope | 13:30 |
mdbooth | Although I think that would be the cleanest fix. Push the is_volume_backed test into snapshot() and have it call the correct implementation after setting a task state | 13:31 |
openstackgerrit | Markus Zoeller (markus_z) proposed openstack/nova: WIP: live-migration test hook for serial console https://review.openstack.org/347471 | 13:31 |
openstackgerrit | Markus Zoeller (markus_z) proposed openstack/nova: libvirt: Acquire TCP ports for console during live migration https://review.openstack.org/275801 | 13:31 |
openstackgerrit | Markus Zoeller (markus_z) proposed openstack/nova: libvirt: fix live-migration with serial console check https://review.openstack.org/335132 | 13:31 |
openstackgerrit | Markus Zoeller (markus_z) proposed openstack/nova: Add functional regression test for bug 1595962 https://review.openstack.org/338416 | 13:32 |
openstack | bug 1595962 in OpenStack Compute (nova) "live migration with disabled vnc/spice not possible" [Undecided,In progress] https://launchpad.net/bugs/1595962 - Assigned to Markus Zoeller (markus_z) (mzoeller) | 13:32 |
*** pwnall has joined #openstack-nova | 13:32 | |
*** zz_dimtruck has joined #openstack-nova | 13:33 | |
*** zz_dimtruck is now known as dimtruck | 13:33 | |
*** leifz has joined #openstack-nova | 13:33 | |
*** ducttape_ has quit IRC | 13:34 | |
*** syjulian_ has quit IRC | 13:35 | |
dansmith | cdent: see my reply to your "what we know" email | 13:37 |
* cdent waits for delivery | 13:37 | |
cdent | ah, there it is, reading | 13:38 |
*** pwnall has quit IRC | 13:38 | |
*** pwnall has joined #openstack-nova | 13:39 | |
*** code-R_ has quit IRC | 13:39 | |
cdent | dansmith: so are you thinking the problem is not in the writing of the allocations, but rather that the scheduler is being "wrong" and we've exposed that problem as a result of the allocation handling? Or, that we're not doing allocation ratio properly in the placement api? | 13:39 |
markus_z | mriedem: FYI: I've put the bugs for these patches on the RC blocker list: https://bugs.launchpad.net/nova/+bugs?field.tag=newton-rc-potential | 13:40 |
mriedem | markus_z: ok | 13:40 |
dansmith | cdent: could be the latter, but the scheduler is definitely not configured to care about overcommitting vcpus, so it's entirely possible that we're over that limit.. I just would have thought setting the ratio to 4.0 would clear it up | 13:40 |
dansmith | cdent: I'm going to poke locally today, but thought you could look as well | 13:40 |
cdent | yeah, can do | 13:40 |
mriedem | mdbooth: this looks like it might be in your wheelhouse https://bugs.launchpad.net/nova/+bug/1597521 | 13:41 |
openstack | Launchpad bug 1597521 in OpenStack Compute (nova) "files should not be injected if config drive is configured" [High,In progress] - Assigned to Vladik Romanovsky (vladik-romanovsky) | 13:41 |
*** links has quit IRC | 13:41 | |
mriedem | mdbooth: are you still good with the patch for that bug? | 13:41 |
mdbooth | Looking... | 13:41 |
mdbooth | mriedem: Yup, still good. | 13:43 |
openstackgerrit | Doug Hellmann proposed openstack/python-novaclient: Update reno for stable/newton https://review.openstack.org/364954 | 13:43 |
*** burt has joined #openstack-nova | 13:44 | |
*** akshai has joined #openstack-nova | 13:45 | |
*** kevinbenton has quit IRC | 13:47 | |
*** su_zhang has joined #openstack-nova | 13:49 | |
mriedem | mdbooth: hmm, would have been nice to just fix the bug in that block of code w/o moving it all out into a separate method | 13:50 |
*** kevinbenton has joined #openstack-nova | 13:50 | |
mriedem | makes it hard to identify what the actual fix is | 13:50 |
mriedem | the refactor should have come on top separately | 13:50 |
*** ecelik has quit IRC | 13:51 | |
cdent | dansmith: a thing I'm noticing just now while watching the logs is that when I create a server it immediately puts the allocations and then puts inventories three times | 13:51 |
*** mmedvede has joined #openstack-nova | 13:52 | |
cdent | which is a bit more than I would have expected, even given that we are being promiscuous | 13:52 |
*** browne has joined #openstack-nova | 13:52 | |
*** nkrinner is now known as nkrinner_afk | 13:53 | |
mriedem | ah i see rpodolyaka said the same thing | 13:54 |
dansmith | cdent: I think it's because of the generation conflicts.. it tries to optimistically calculate the next generation when it does one inventory | 13:54 |
mriedem | don't mingle refactors with bug fixes | 13:54 |
dansmith | cdent: when I wrote that I didn't realize we'd be bumping it during allocation too | 13:54 |
*** gouthamr_ has quit IRC | 13:54 | |
*** su_zhang has quit IRC | 13:55 | |
dansmith | cdent: I'm stacking now with all current stuff, in a bit I'll see if I see that pattern exactly too | 13:55 |
*** su_zhang has joined #openstack-nova | 13:55 | |
*** gouthamr has joined #openstack-nova | 13:55 | |
cdent | I'm only seeing 200 status, no 409, which is what would be required if it was generation conflict? | 13:55 |
* cdent checks the logging code | 13:55 | |
dansmith | oh? | 13:56 |
cdent | grep PUT placement-api.log |grep inventories |grep -v 'status: 200' has a null result | 13:57 |
*** sameo_ has joined #openstack-nova | 13:57 | |
dansmith | I'm just now building a devstack with the logging fixes so I haven't see any of that yet | 13:57 |
*** sameo has quit IRC | 13:58 | |
mdbooth | mriedem: Also sfinucan and I both agreed :) | 13:58 |
mdbooth | However, I don't have a code fairy and the job's already done | 13:58 |
mdbooth | So I wasn't bothered enough for a -1 | 13:59 |
* cdent experiments som more | 13:59 | |
mriedem | what would a code fairy do? | 13:59 |
mdbooth | The code fairy would split it in 2 and get 4 +2s instead of 2 | 13:59 |
mriedem | i think it's just bad practice, refactors shouldn't be mixed in with bug fixes | 13:59 |
*** su_zhang has quit IRC | 13:59 | |
dansmith | cdent: we're putting three inventories but in one go.. any chance the logging code is improperly counting that as three requests? | 14:00 |
mriedem | because it's harder to review and could lead to introducing new bugs | 14:00 |
dansmith | cdent: like if the one target calls the other targets for the list? | 14:00 |
*** su_zhang has joined #openstack-nova | 14:00 | |
*** anush__ has joined #openstack-nova | 14:00 | |
cdent | this is supposed to be per request logging | 14:00 |
dansmith | well, I know | 14:00 |
dansmith | just brainstorming :) | 14:00 |
*** hshiina has joined #openstack-nova | 14:01 | |
*** markus_z has quit IRC | 14:01 | |
cfriesen | question for the microversion folks...if we were to change the accounting of hypervisor vcpus_used for SMT hosts when instances use the ISOLATE cpu thread policy, would that require a microversion bump since it's user-visible behaviour change? | 14:01 |
cdent | dansmith: the logging's not right, it is only recording success | 14:01 |
cdent | i'll figure out how to fix and send up a patch | 14:02 |
dansmith | ah | 14:02 |
cdent | how, the fact that it is getting three successul puts is still true | 14:02 |
*** mat128 is now known as mat128|away | 14:02 | |
*** tesseract- has quit IRC | 14:03 | |
*** code-R has joined #openstack-nova | 14:04 | |
bauwser | dansmith: so, when you say the scheduler doesn't use the CoreFilter, you mean the job ? | 14:04 |
*** gszasz has joined #openstack-nova | 14:05 | |
dansmith | bauwser: yeah | 14:05 |
bauwser | I'm surprised tho | 14:05 |
bauwser | because I thought CoreFilter was a default one | 14:05 |
dansmith | bauwser: dunno, but we override the list | 14:05 |
bauwser | oh snap... | 14:05 |
*** kaisers_ has quit IRC | 14:05 | |
dansmith | bauwser: also, I want to run mine locally and make sure that my database is getting the value from the config, because we set the allocation ratio from the config on the compute node, and then use the compute node to inform the placement request | 14:06 |
cdent | dansmith: ah, worse than I thought, it is logging success (200) when it's not success. this will require some fairly extensive jiggery pokery. so for the time being I wouldn't rely on the logged web requests at all, just go with the logs that the resource tracker is doing | 14:06 |
bauwser | dansmith: I need to update my devstack | 14:06 |
bauwser | and I'll see | 14:06 |
dansmith | cdent: lol, okay | 14:07 |
dansmith | cdent: well, the vcpu over cap thing is from the RT anyway | 14:07 |
*** ducttape_ has joined #openstack-nova | 14:08 | |
*** trisq has joined #openstack-nova | 14:08 | |
*** jamesdenton has joined #openstack-nova | 14:09 | |
*** mtanino has joined #openstack-nova | 14:10 | |
*** rama_y has joined #openstack-nova | 14:10 | |
*** mingyu has quit IRC | 14:11 | |
*** tesseract- has joined #openstack-nova | 14:13 | |
dansmith | ahh | 14:15 |
dansmith | bauwser: I forgot about how 0.0 means 16.0 for cpu_allocation :) | 14:15 |
dansmith | so, I'm sure we're not over 4*16 cpus | 14:16 |
bauwser | 0.0 signals "read the config" | 14:16 |
dansmith | which means something must be actually wrong | 14:16 |
dansmith | bauwser: NOTE: This can be set per-compute, or if set to 0.0, the value | 14:16 |
dansmith | set on the scheduler node(s) or compute node(s) will be used | 14:16 |
dansmith | and defaulted to 16.0'. | 14:16 |
dansmith | oh wait wait | 14:16 |
dansmith | that's ram huh | 14:16 |
bauwser | dansmith: ram ratio is 1.5 | 14:16 |
bauwser | dansmith: but I need to understand more the issue | 14:16 |
bauwser | dansmith: do you have some trace I could look | 14:17 |
dansmith | bauwser: https://github.com/openstack/nova/blob/master/nova/conf/compute.py#L207 | 14:17 |
dansmith | no trace | 14:17 |
bauwser | because I have hard time figuring out what is the problem | 14:17 |
rpodolyaka | jroll: hey! could you please take a look at https://review.openstack.org/#/q/status:open+project:openstack/nova+branch:master+topic:bug/1606496 when you have a minute? | 14:17 |
dansmith | yeah, so 16 is for cpu | 14:17 |
bauwser | (and running a meeting atm makes me kinda off) | 14:17 |
dansmith | conf is hard to read | 14:17 |
bauwser | 16 is default for cpu, 1.5 for ram and 1.0 for disk | 14:17 |
jroll | rpodolyaka: yes, still on my list, will do it today :) | 14:18 |
rpodolyaka | jroll: thank you! | 14:18 |
dansmith | bauwser: right, that's what I said above | 14:18 |
*** marst has quit IRC | 14:19 | |
*** ducttape_ has quit IRC | 14:19 | |
*** kaisers_ has joined #openstack-nova | 14:19 | |
dansmith | cdent: yeah, so anyway, allocation ratio is 16.0 for cpu in my database by default, and 4 vcpus on the gate workers, so we really shouldn't be overcommitted | 14:19 |
dansmith | cdent: so something is up in that allocation calculation | 14:19 |
cdent | dansmith: yeah. I've become distracted by this logging thing so haven't had a chance to look in any detail yet | 14:20 |
*** hshiina has quit IRC | 14:22 | |
*** moshele has joined #openstack-nova | 14:25 | |
mriedem | dansmith: are you going to openstack days seattle? | 14:26 |
*** annegentle has joined #openstack-nova | 14:26 | |
dansmith | mriedem: wasn't planning on it | 14:26 |
*** diogogmt has joined #openstack-nova | 14:28 | |
*** diablo_rojo has joined #openstack-nova | 14:29 | |
mriedem | diablo_rojo: yo | 14:29 |
mriedem | so, | 14:29 |
mriedem | diablo_rojo: auggy is in portland | 14:30 |
mriedem | and is the nova mentoring person | 14:30 |
*** prateek_ has joined #openstack-nova | 14:30 | |
mriedem | diablo_rojo: so if you're looking for someone to go to seattle and talk about new contributors to nova, she'd be a good it | 14:30 |
mriedem | *fit | 14:30 |
*** moshele has quit IRC | 14:30 | |
pumaranikar | alex_xu, can you please check my comment on https://review.openstack.org/#/c/342299/ | 14:30 |
*** mjura has quit IRC | 14:31 | |
diablo_rojo | mriedem, works for me. | 14:31 |
diablo_rojo | Anyone else going to be at OpenStack Days Seattle? | 14:31 |
mriedem | clarkb: ^? | 14:31 |
diablo_rojo | clarkb, is probably going to go. Chatted with him a bit yesterday about it. | 14:32 |
diablo_rojo | dansmith, will you be going to OSD Seattle? | 14:32 |
dansmith | diablo_rojo: no | 14:33 |
*** andymaier has quit IRC | 14:33 | |
diablo_rojo | Shoot. Okay. | 14:33 |
dansmith | cdent: also, with four instances being updated every time we refresh the resource tracker, I think we never get to update inventory again because of generation clashes | 14:33 |
dansmith | cdent: which we could avoid if we could look at allocations | 14:33 |
*** prateek has quit IRC | 14:33 | |
*** su_zhang has quit IRC | 14:34 | |
*** su_zhang has joined #openstack-nova | 14:35 | |
*** RuiChen has quit IRC | 14:36 | |
dansmith | cdent: hmm, also seem to be getting the same generation back when we re-poll the resource provider after a generation clash | 14:37 |
bauwser | is it me or Gerrit is in the weeds ? | 14:38 |
*** zenoway has quit IRC | 14:38 | |
cdent | dansmith: i made a bug for the logging thing, maybe you could make bugs for these things too because I'm not really able to keep track of you. Not that I want to kill the conversation, just that I worry things might get lost. | 14:38 |
*** eharney has quit IRC | 14:38 | |
*** zenoway has joined #openstack-nova | 14:38 | |
cdent | On the reading allocations thing, that's something I reckon we'll need jay involved in the conversation about. It's a bigger change than I would feel comfortable saying go on without more brains. | 14:39 |
*** ducttape_ has joined #openstack-nova | 14:39 | |
*** su_zhang has quit IRC | 14:39 | |
dansmith | yeah | 14:39 |
*** trisq has quit IRC | 14:41 | |
*** zenoway has quit IRC | 14:42 | |
bauwser | dansmith: cdent: worth trying to write a patch for updating allocations when migrating? | 14:42 |
*** zenoway has joined #openstack-nova | 14:42 | |
openstackgerrit | Chris Dent proposed openstack/nova: Move placement api request logging to middleware https://review.openstack.org/365015 | 14:42 |
bauwser | it's on my bag, and I feel it can be a bugfix | 14:42 |
dansmith | bauwser: well, maybe, let me figure out why this is happening first before we pile more on here | 14:43 |
cdent | dansmith: that ^ is logging that logs 409s (and others) properly | 14:43 |
bauwser | dansmith: sorry, Gerrit is awfully slow for me today, so I'm having hard time to look at your devstack patch | 14:44 |
*** links has joined #openstack-nova | 14:47 | |
*** EinstCrazy has joined #openstack-nova | 14:47 | |
*** tjones has quit IRC | 14:48 | |
dansmith | bauwser: devstack patch is not needed anymore | 14:48 |
*** tjones has joined #openstack-nova | 14:49 | |
bauwser | dansmith: I'm like on dotted support given our meeting, so sorry :( | 14:49 |
*** rrecio has joined #openstack-nova | 14:50 | |
cdent | dansmith: have you got a cookbook for triggering the problems? I lowered my cpu allocation ratio down low and all was cool: server creates started failing. I guess I'll raise it back up and see if I can get allocations to fail? | 14:51 |
dansmith | I'm a couple levels down the rabbit hole | 14:52 |
openstackgerrit | Takashi NATSUME proposed openstack/nova-specs: Fix blueprints and specs link in ocata template https://review.openstack.org/335312 | 14:52 |
dansmith | but yeah, something like that | 14:52 |
*** EinstCrazy has quit IRC | 14:52 | |
dansmith | cdent: I'm definitely having some sort of issue, by the way | 14:53 |
bauwser | dansmith: just to make it clear, the problem you see is because you think we don't check the allocation ratios ? | 14:53 |
*** EinstCrazy has joined #openstack-nova | 14:53 | |
dansmith | cdent: I'm polling the rp, getting the generation, then sending a new inventory with that generation, failing, sleeping 2 sec, repeat, and I get the same generation reported from the server | 14:54 |
*** rrecio_ has joined #openstack-nova | 14:54 | |
dansmith | cdent: so I'm getting a 409 for some other reason | 14:54 |
cdent | dansmith: do you have any inventory? | 14:54 |
dansmith | cdent: perhaps the server refuses to let me update my inventory if it would put me into the red on capacity? | 14:54 |
dansmith | cdent: you mean allocations? | 14:54 |
*** EinstCrazy has quit IRC | 14:55 | |
cdent | Is the 409 happening when you try to write allocations or inventory? | 14:55 |
dansmith | cdent: inventory | 14:55 |
melwitt | bauwser: I'm getting 502s from gerrit too | 14:55 |
bauwser | melwitt: I reported to -infra | 14:56 |
*** _ducttape_ has joined #openstack-nova | 14:56 | |
melwitt | cool, thanks | 14:56 |
*** _ducttape_ has quit IRC | 14:56 | |
*** _ducttape_ has joined #openstack-nova | 14:56 | |
cdent | dansmith: in that context a 409 can happen either when somebody else did an update (for example an allocation was written) or when the inventory you are trying to write has a logic conflict (reserved greater than total or allocation_ratio blowing things wacky) | 14:56 |
*** rrecio has quit IRC | 14:57 | |
dansmith | cdent: it's not the former because the generation hasn't changed when Irepoll | 14:57 |
cdent | are you logging the contents of inventory? | 14:58 |
dansmith | cdent: what if I write an inventory that would mean my allocations are over budget? | 14:58 |
dansmith | yes | 14:58 |
* cdent looks at the code | 14:58 | |
dansmith | lets hangout in a few when I get off this call | 14:58 |
bauwser | cdent: we can get 409s when we exceed capacity, right? | 14:58 |
cdent | bauwser: yes | 14:58 |
*** tbachman has quit IRC | 14:58 | |
dansmith | bauwser: for allocations, but that's not this | 14:58 |
bauwser | cdent: ie. when putting an allocation that would go over the total capacity | 14:59 |
bauwser | dansmith: ah ok | 14:59 |
cdent | dansmith: if allocations and allocations['usage'] > inv_record.capacity: | 14:59 |
*** ducttape_ has quit IRC | 14:59 | |
* bauwser is puzzled and feels frustrated by not being able to put the level of attention he should | 14:59 | |
cdent | so yeah, if you have allocations above the new inventory things go wrong | 14:59 |
dansmith | cdent: that's a problem I think, | 14:59 |
dansmith | because I can cause that by changing config | 14:59 |
dansmith | and then the compute node would be hamstrung, if we were using this for the main path | 15:00 |
dansmith | cdent: https://hangouts.google.com/call/7iyntj4ldveepnujkafit74g7ie | 15:00 |
cdent | dansmith: give me couple minutes to coffee up? | 15:00 |
dansmith | okay | 15:00 |
dansmith | will do the same | 15:00 |
*** rcernin has quit IRC | 15:01 | |
cdent | while the kettle boils, if you change config, when you restart n-cpu to read that config, would/should the existing allocations get reset? and what should the failure mode be when config'd constraints and existing vms conflict (which is orthogonal to the placement api but a thing I'd like to know)? | 15:03 |
*** links has quit IRC | 15:03 | |
*** mlavalle has joined #openstack-nova | 15:04 | |
johnthetubaguy | dansmith: I bet those inventories just send 0.0, I missed that | 15:04 |
dansmith | johnthetubaguy: they don't | 15:04 |
dansmith | cdent: um, what? | 15:04 |
dansmith | cdent: lets just talk voice as I think it'll be easier | 15:05 |
johnthetubaguy | sorry, I was only half way through scroll back, and send that by accieden | 15:05 |
*** salv-orlando has quit IRC | 15:06 | |
*** rama_y has quit IRC | 15:06 | |
*** hemnafk is now known as hemna | 15:08 | |
*** paul-carlton has joined #openstack-nova | 15:11 | |
*** artom has quit IRC | 15:15 | |
*** pcaruana has quit IRC | 15:15 | |
*** bnemec is now known as beekneemech | 15:16 | |
*** salv-orlando has joined #openstack-nova | 15:16 | |
*** sdake has joined #openstack-nova | 15:16 | |
mriedem | no instance actions recorded for a swap volume, is that a bug? | 15:17 |
mriedem | it's an orchestrated thing with cinder volume migration/retype and async | 15:18 |
mriedem | seems like it should have instance actions | 15:18 |
mriedem | lascii: ^? | 15:18 |
*** matrohon has quit IRC | 15:18 | |
mriedem | wtf is superdan? | 15:18 |
*** dansmith is now known as superdan | 15:19 | |
johnthetubaguy | mriedem: we have loads of actions missing instance actions AFAIK | 15:20 |
johnthetubaguy | mriedem: there was a spec to add them, but it didn't make the deadline | 15:21 |
mriedem | i know we had some approved blueprints for some random things | 15:21 |
mriedem | but i might be thinking about notifications | 15:21 |
johnthetubaguy | possibly, the OSIC folks could work on that kind of stuff, we just didn't get the spec approved in time | 15:22 |
-openstackstatus- NOTICE: The Gerrit service on review.openstack.org is restarting quickly to relieve resource pressure and restore normal performance | 15:22 | |
*** marst has joined #openstack-nova | 15:22 | |
johnthetubaguy | will try push on next cycle (adds it too the list) | 15:22 |
*** mingyu has joined #openstack-nova | 15:22 | |
mriedem | i'm never sure how much we're trying to fill those gaps or holding off for a new thing like tasks | 15:24 |
mriedem | johnthetubaguy: was this the spec? https://review.openstack.org/#/c/256743/6/specs/newton/approved/expand-instance-actions-coverage.rst | 15:25 |
johnthetubaguy | mriedem: yeah, I remember the disagreement on the way forward now | 15:26 |
*** whenry has joined #openstack-nova | 15:29 | |
cdent | superdan, johnthetubaguy, mriedem can some of you gander at https://review.openstack.org/#/c/365015/ ? It fixes the logging on the placement api and having that as part of the current world of fixing stuff would be handy | 15:33 |
*** code-R has quit IRC | 15:33 | |
cdent | superdan: i'm going to base these new changes off that (so I can actually have the logs) | 15:33 |
*** Dinesh_Bhor has quit IRC | 15:34 | |
superdan | cdent: is requestlog.py roughly what sdague had added yesterday? it's a little different but hard to tell how much | 15:35 |
bauwser | cdent: FWIW, I downed the priority of the bug to High given that's not impacting our existing deployments | 15:35 |
bauwser | we can't really see it's critical :/ | 15:35 |
bauwser | say$ | 15:35 |
cdent | bauwser: thanks, I wasn't savvy to the what it means, I was mostly just clicking on things...but now I know. | 15:36 |
cdent | superdan: what it does is the same, what's different is when it happens | 15:36 |
cdent | (and that it works) | 15:36 |
*** kaisers_ has quit IRC | 15:36 | |
superdan | cdent: okay it's just structured differently | 15:36 |
cdent | yes | 15:36 |
bauwser | cdent: when is the logging made in the compute api ? AFAIR, that's a paste middleware, right? | 15:37 |
* bauwser checks paste.ini | 15:37 | |
mriedem | this isn't in paste.ini | 15:38 |
cdent | bauwser: I don't know. I did this as a fast fix becaue the existing logs were completely wrong and we needed accurate logs to diagnose today's behaviors | 15:38 |
*** paul-carlton has quit IRC | 15:38 | |
*** sridharg has quit IRC | 15:38 | |
mriedem | i'll wait for CI results of the placement job to see the different | 15:39 |
mriedem | *difference | 15:39 |
mriedem | so today the 404s and 409s aren't showing up, but with this they should? | 15:39 |
cdent | mriedem: yes | 15:39 |
johnthetubaguy | I guess they error-ed before the logs code would catch it before? | 15:40 |
mriedem | hmm, i was looking for req-cd944345-4f01-487f-8a75-764e70824302 in http://logs.openstack.org/98/364498/5/check/gate-tempest-dsvm-neutron-placement-full-ubuntu-xenial-nv/acb2458/logs/apache/placement-api.txt.gz but it's not there | 15:41 |
mriedem | but it is here http://logs.openstack.org/98/364498/5/check/gate-tempest-dsvm-neutron-placement-full-ubuntu-xenial-nv/acb2458/logs/screen-n-cpu.txt.gz?level=TRACE#_2016-09-01_21_43_14_607 | 15:41 |
*** andymaier has joined #openstack-nova | 15:42 | |
*** tesseract- has quit IRC | 15:42 | |
*** rama_y has joined #openstack-nova | 15:43 | |
bauwser | cdent: given my devstack is still not pimped for the placement API (I need a bit more time), could you please verify whether all of that isn't just due to the fact of https://review.openstack.org/#/c/365015/1/nova/api/openstack/placement/handler.py | 15:44 |
bauwser | ? | 15:44 |
cdent | which fact? | 15:44 |
cdent | oh i see it now | 15:45 |
cdent | no, it's not just that | 15:45 |
cdent | if there is an error anywhere else in the middleware stack it won't get logged | 15:45 |
bauwser | cdent: for the middleware stack, I agree | 15:45 |
cdent | the logging _has_ to be on the outside of the middleware, otherwise things will not be logged | 15:45 |
bauwser | cdent: but the bug mentions https://bugs.launchpad.net/nova/+bug/1619690 | 15:46 |
openstack | Launchpad bug 1619690 in OpenStack Compute (nova) "request logging in placement api always logs success" [High,In progress] - Assigned to Chris Dent (cdent) | 15:46 |
bauwser | cdent: which is fairly different | 15:46 |
* auggy 's ears are burning | 15:46 | |
cdent | yes, the reason for constant success is because some of the time resp hasn't been set | 15:46 |
bauwser | cdent: there are 2 problems IMHO, #1 is logging with 200 everytime, #2 being that we don't log if middleware expects prblems | 15:46 |
*** _ducttape_ has quit IRC | 15:46 | |
cdent | and it has a default | 15:46 |
bauwser | mmm, ok | 15:47 |
*** aghadge has joined #openstack-nova | 15:47 | |
cdent | so by moving up the stack a bit we are always using the _real_ response | 15:47 |
mriedem | scottda: your tempest test is for retype that will test nova swap_volume correct? | 15:47 |
mriedem | scottda: but the tempest test only runs if we have multiple cinder backends defined in tempest.conf? | 15:47 |
cdent | bauwser: I'm sorry I left out some details but I was rushed: I needed actually working logging! | 15:47 |
auggy | diablo_rojo: hi! i'm in pdx, i wasn't planning on going to openstack days seattle because i actually didn't know about it | 15:47 |
mriedem | scottda: which happens via devstack flag, which is set in a new job definition of yours right? | 15:47 |
auggy | diablo_rojo: but now that i know about it, i am going to see about going :) | 15:48 |
diablo_rojo | auggy, Cool :) I am fairly sure I will be there so if you do end up going, give me a heads up and we can meetup while we are both there. | 15:49 |
auggy | diablo_rojo: will do :) | 15:49 |
*** sridharg has joined #openstack-nova | 15:51 | |
diablo_rojo | auggy, Hopefully see you there! | 15:51 |
*** eharney has joined #openstack-nova | 15:52 | |
*** rmart04 has quit IRC | 15:52 | |
*** ducttape_ has joined #openstack-nova | 15:52 | |
*** nic has joined #openstack-nova | 15:56 | |
*** matrohon has joined #openstack-nova | 15:56 | |
*** praneshp has joined #openstack-nova | 15:57 | |
*** raj_singh_ has joined #openstack-nova | 15:57 | |
*** pilgrimstack has quit IRC | 15:58 | |
*** matrohon has quit IRC | 15:58 | |
*** matrohon has joined #openstack-nova | 15:58 | |
*** raj_singh_ has quit IRC | 16:01 | |
*** trisq has joined #openstack-nova | 16:02 | |
mriedem | scottda: have you setup a d-g change to test the new job and the tempest test? | 16:02 |
mriedem | to see it passing? | 16:02 |
*** tongli has quit IRC | 16:02 | |
mriedem | scottda: oh i guess that's what this is for https://review.openstack.org/#/c/331840/ | 16:04 |
superdan | mriedem: so in case you didn't follow, we've definitely got some things we need to fix server and client side on the placement stuff | 16:05 |
superdan | mriedem: one of them is a slight conceptual change so we probably want to wait to merge it until jay can look and bless it | 16:05 |
superdan | but we're going to get going on it and try to have it ready for next week | 16:05 |
mriedem | ok, yeah i haven't been following, except i saw the logging change and was waiting to see the results of that | 16:06 |
mriedem | i'm going to try and check out for the weekend in about an hour | 16:07 |
superdan | okay | 16:07 |
superdan | I'm going to fade early myself | 16:07 |
bauwser | FWIW, I'm deploying a placement-api pimped devstack | 16:07 |
*** paul-carlton has joined #openstack-nova | 16:07 | |
bauwser | so, I could test a few bits | 16:07 |
*** inara has quit IRC | 16:08 | |
bauwser | superdan: glad you spotted that bug before we go further | 16:08 |
*** inara has joined #openstack-nova | 16:08 | |
superdan | bauwser: cool and yeah :) | 16:08 |
*** raunak has joined #openstack-nova | 16:08 | |
bauwser | superdan: I used the word "source of truth" when I was talking of compute node, and I know it's a bit confusing | 16:09 |
auggy | superdan: bauwser if there's anything i can help out with on the placement stuff, let me know, i've just been reading through what's happening and wrapping my head around it | 16:09 |
auggy | apparantly i'm pretty good at mox tests now | 16:09 |
superdan | auggy: you and me both | 16:09 |
bauwser | superdan: but what I wanted to express is that we need somehow to trust the compute node for providing the right view of what it runs | 16:10 |
superdan | bauwser: yeah and I'm not sure that's the right thing to do | 16:10 |
bauwser | I know | 16:11 |
superdan | bauwser: but, we can trust anyone that puts a new allocation that isn't a change, IMHO, which is all we need | 16:11 |
mriedem | auggy: did you ever get a change pushed up to gitignore etc/nova/policy.yaml.merged? | 16:11 |
auggy | mriedem: yes i pushed that change | 16:11 |
*** mingyu has quit IRC | 16:11 | |
mriedem | link? | 16:11 |
auggy | mriedem: i'll find the magic link | 16:11 |
bauwser | superdan: I know that you have concerns with the compute node being the one which should get the right view of the world | 16:11 |
*** rlrossit has quit IRC | 16:12 | |
auggy | mriedem: https://review.openstack.org/#/c/363768/ | 16:12 |
scottda | mriedem: Yes to all of the above. And MattTR suggested adding that test to tempest experimental job as well. | 16:12 |
auggy | mriedem: i think i pinged you after i pushed it but with all the crazy happening i reckon it was missed ;) | 16:12 |
auggy | aw sfinucan +1'ed it :) | 16:13 |
*** tonytan4ever has joined #openstack-nova | 16:13 | |
openstackgerrit | Matt Riedemann proposed openstack/nova: DNM: test swap-volume fix https://review.openstack.org/365050 | 16:13 |
mriedem | scottda: you mean add the gate-tempest-dsvm-lvm-multibackend-nv job to tempest experimental right? | 16:14 |
sfinucan | auggy: thank my nova review dash ;) | 16:14 |
mriedem | scottda: that's a good idea | 16:14 |
openstackgerrit | Ludovic Beliveau proposed openstack/nova: Update available resources before confirm stage https://review.openstack.org/327356 | 16:14 |
scottda | mriedem: yes | 16:14 |
auggy | sfinucan: yeah i need to be better about reviews, they just take me a long time and sometimes they merge before i'm done getting through it | 16:14 |
mriedem | scottda: ok that patch ^ above is also using it to test a nova fix for swap volume | 16:15 |
mriedem | scottda: the bug being you can swap from vol A to B but can't swap back from B to A | 16:15 |
mriedem | nova blows up | 16:15 |
*** zenoway has quit IRC | 16:16 | |
*** andymaier has quit IRC | 16:16 | |
*** rlrossit has joined #openstack-nova | 16:17 | |
*** zenoway has joined #openstack-nova | 16:17 | |
*** Swami has joined #openstack-nova | 16:17 | |
*** armax has joined #openstack-nova | 16:18 | |
*** tonytan_brb has joined #openstack-nova | 16:19 | |
Swami | mriedem: Matt could you please review this nova-neutron patch. https://review.openstack.org/#/c/275073/40 | 16:19 |
*** kfarr has joined #openstack-nova | 16:20 | |
mriedem | i can only review that change once per quarter | 16:20 |
*** matrohon has quit IRC | 16:20 | |
Swami | mriedem: no kidding | 16:21 |
mriedem | Swami: will try to get it in next week | 16:21 |
mriedem | i'm quickly running out of steam to do anything more this week | 16:21 |
Swami | mriedem: thanks | 16:21 |
Swami | mriedem: no worries. | 16:21 |
*** oomichi_ has joined #openstack-nova | 16:21 | |
*** zenoway has quit IRC | 16:22 | |
*** tonytan4ever has quit IRC | 16:22 | |
*** zul has quit IRC | 16:23 | |
*** dims has quit IRC | 16:23 | |
bauwser | hurrah, my devstack works | 16:25 |
*** tongli has joined #openstack-nova | 16:28 | |
auggy | mriedem: also i had gotten this one done but hadn't checked the gate status and apparantly i can't yaml haha https://review.openstack.org/#/c/363803/ | 16:28 |
auggy | anyways that's fixed and i'm just seeing what these two failures are about, i thought it was good to go | 16:29 |
mriedem | looks like infra | 16:29 |
mriedem | the job aborted halfway through | 16:29 |
mriedem | just recheck it | 16:30 |
openstackgerrit | Brent Eagles proposed openstack/nova: Add bigswitch command to compute rootwrap filters https://review.openstack.org/358024 | 16:31 |
*** fragatina has joined #openstack-nova | 16:32 | |
*** tongli has quit IRC | 16:32 | |
*** dims has joined #openstack-nova | 16:32 | |
superdan | cdent: um, I think the api exposes and accepts inventories in different formats | 16:33 |
cdent | superdan say what? | 16:33 |
superdan | hang on | 16:33 |
*** parora has joined #openstack-nova | 16:33 | |
*** prateek_ has quit IRC | 16:34 | |
superdan | well, I can't provide you a link because the gabbits aren't exhaustive | 16:34 |
auggy | mriedem: yeah that's what it was looking like to me too, rechecking away | 16:34 |
superdan | cdent: however, when I put a list of inventories, the resource_provider_generation is a toplevel thing, | 16:35 |
superdan | cdent: however in the response it looks like it is in each inventory by resource class | 16:35 |
superdan | cdent: which means if I have no inventories yet, I don't know what the generation is unless I look at the resource provider itself | 16:35 |
cdent | superdan: there isn't get put symmetry | 16:35 |
superdan | cdent: and it means that the value is repeated over and over for the same resource provider once for each inventory record | 16:36 |
*** zul has joined #openstack-nova | 16:36 | |
superdan | cdent: why not? isn't that unrestful? | 16:36 |
*** fragatina has quit IRC | 16:36 | |
clarkb | mriedem: auggy if you pull up the ansible log for that job it lost ssh connectivity to the host. Jeblair was looking into that and root cause seems to be Internets so we need to work on making things retry better | 16:36 |
cdent | superdan: when I tried to make the placement api restful, jay got annoyed... | 16:36 |
superdan | ffs | 16:36 |
cdent | the version of the api that currently exists is the third design | 16:36 |
*** Apoorva has joined #openstack-nova | 16:36 | |
*** fragatina has joined #openstack-nova | 16:36 | |
auggy | clarkb: ah cool, yeah that's what i was poking around to see | 16:36 |
armax | mriedem: ping | 16:37 |
cdent | there was jay's first one which i then made restful, which he then switched to omething else | 16:37 |
cdent | at which point I gave up fighting | 16:37 |
superdan | I'm not really interested in who is at fault I just want to make this work | 16:37 |
superdan | so whatever, I guess I will just do three http calls each time | 16:37 |
cdent | I'm merely trying to provide context and history | 16:37 |
auggy | clarkb: i sometimes wonder about your list of highlighted words... haha you seem to magically appear right when you are able to offer some great insight | 16:37 |
cdent | three? | 16:37 |
*** ftersin has left #openstack-nova | 16:38 | |
clarkb | auggy: I like to investigate when someone says "probably infra just recheck" :) | 16:38 |
superdan | cdent: yes, one to reliably get the current generation, one to get the current inventory and one to put the new inventory (if needed) .. so two always, three if there is an update | 16:38 |
mriedem | and i was right :) | 16:38 |
mriedem | armax: pong | 16:38 |
clarkb | mriedem: yup | 16:38 |
* auggy thinks clarkb is secretly a heinzelmännchen | 16:38 | |
jeblair | auggy: i just follow clarkb around since he has the highlight game covered, apparently | 16:39 |
armax | mriedem: we got stung by the os-vif bug again here: http://logs.openstack.org/14/365014/1/check/gate-tempest-dsvm-neutron-linuxbridge/5c3eecd/ | 16:39 |
auggy | jeblair: ha | 16:39 |
armax | mriedem: http://logs.openstack.org/14/365014/1/check/gate-tempest-dsvm-neutron-linuxbridge/5c3eecd/logs/screen-n-cpu.txt.gz?level=TRACE#_2016-09-02_15_04_53_014 | 16:39 |
armax | mriedem: I had an E-R query for this | 16:39 |
mriedem | armax: is that because the fix isn't released? | 16:39 |
armax | mriedem: yeah I think so | 16:39 |
armax | but I am curious as to why E-R didn’t pick it up | 16:40 |
cdent | superdan: don't you have a local reference to the resource provider that you can keep updated as you make successful changes, and just revamp that when you get a fail? | 16:40 |
cdent | sorry I used the word "just" that's a no no | 16:40 |
armax | mriedem: looking closer the query doesn’t seem to match | 16:40 |
openstackgerrit | Tuan Luong-Anh proposed openstack/nova: Python 3.x compatibility modules urllib https://review.openstack.org/361964 | 16:40 |
superdan | cdent: yeah, but it's just getting kinda messy to keep stabbing things into the local cache | 16:40 |
*** paul-carlton has quit IRC | 16:41 | |
cdent | i had assumed that one was already there | 16:41 |
*** paul-carlton has joined #openstack-nova | 16:41 | |
mriedem | armax: is the LB fix in os-vif on master released at all? | 16:41 |
armax | mriedem: yes | 16:41 |
armax | ah | 16:41 |
armax | no | 16:41 |
armax | I don’t think so | 16:41 |
mriedem | armax: the stable/newton is approved at least now https://review.openstack.org/#/c/363975/ | 16:41 |
armax | mriedem: ack | 16:41 |
armax | mriedem: question I had for you | 16:41 |
armax | was | 16:41 |
mriedem | armax: and yes https://review.openstack.org/#/c/361516/ isn't released | 16:42 |
mriedem | so that's why you're still failing | 16:42 |
armax | on https://review.openstack.org/#/c/361458/ | 16:42 |
mriedem | we could do a patch release....to get u-c updated | 16:42 |
armax | mriedem: shall I replace the InterfaceAttachFailed error with ‘can't create bridge with the same name’ | 16:42 |
armax | mriedem: because for whatever reason this last failure seems to be induced by the same error, but it shows up differently | 16:43 |
openstackgerrit | Tuan Luong-Anh proposed openstack/nova: Python 3.x compatibility modules urllib https://review.openstack.org/361964 | 16:44 |
mriedem | armax: you can do an OR with the messages | 16:44 |
mriedem | to hit both | 16:44 |
armax | mriedem: ack | 16:44 |
armax | mriedem: tweak coming right up | 16:44 |
mriedem | (message:"InterfaceAttachFailed: Failed to attach network adapter device to" OR message:"can't create bridge with the same name") AND tags:screen-n-cpu.txt | 16:45 |
*** csd_ has joined #openstack-nova | 16:45 | |
mriedem | oh different file | 16:46 |
mriedem | oh nvm | 16:46 |
*** annegentle has quit IRC | 16:47 | |
armax | mriedem: https://review.openstack.org/#/c/365064/ | 16:47 |
armax | mriedem: what do you mean differnt file? | 16:48 |
mriedem | ignore | 16:48 |
mriedem | i thought one was checking n-cpu and the other n-net | 16:48 |
mriedem | but they are both messages in n-cpu | 16:48 |
mriedem | i get 24 hits in logstash for (message:"InterfaceAttachFailed: Failed to attach network adapter device to" OR message:"can't create bridge with the same name") AND tags:screen-n-cpu.txt over 7 days | 16:49 |
mriedem | check and gate | 16:49 |
mriedem | 1. gate-tempest-dsvm-neutron-linuxbridge 18 2. gate-tempest-dsvm-neutron-full-ubuntu-xenial 3 3. gate-tempest-dsvm-neutron-multinode-full | 16:49 |
mriedem | they aren't all LB jobs | 16:49 |
armax | mriedem: ack | 16:49 |
*** tongli has joined #openstack-nova | 16:49 | |
*** su_zhang has joined #openstack-nova | 16:50 | |
*** Oku_OS is now known as Oku_OS-away | 16:51 | |
*** sudipto has quit IRC | 16:53 | |
*** sudipto_ has quit IRC | 16:53 | |
*** tongli has quit IRC | 16:54 | |
*** lucasagomes is now known as lucas-dinner | 16:54 | |
*** paul-carlton has quit IRC | 16:54 | |
*** sudipto_ has joined #openstack-nova | 16:56 | |
*** sudipto has joined #openstack-nova | 16:56 | |
*** ZZelle has quit IRC | 17:00 | |
*** ssurana has joined #openstack-nova | 17:00 | |
*** anush__ has quit IRC | 17:00 | |
*** ZZelle has joined #openstack-nova | 17:00 | |
*** amotoki has quit IRC | 17:01 | |
*** mamandle has joined #openstack-nova | 17:04 | |
*** oanson has quit IRC | 17:05 | |
*** trisq has quit IRC | 17:05 | |
*** raj_singh_ has joined #openstack-nova | 17:05 | |
*** unicell1 has quit IRC | 17:05 | |
*** paul-carlton has joined #openstack-nova | 17:06 | |
*** trisq has joined #openstack-nova | 17:06 | |
cdent | superdan: I've got a working version of "don't write a same allocation to the database" but I've got a question: | 17:07 |
cdent | do you need a different response code between "we wrote it" and "it's the same so we didn't write it"? | 17:07 |
figleaf | cdent: My take on that is that the client should never care about the difference | 17:09 |
cdent | figleaf: yeah, in the general case, but in this case we've been having some weird... | 17:10 |
figleaf | cdent: and what if you send 4 allocations, and 1 is new? Or 2? Or all 4? DO those all get different codes? | 17:10 |
cdent | figleaf: the context on this is https://bugs.launchpad.net/nova/+bug/1619723 | 17:11 |
openstack | Launchpad bug 1619723 in OpenStack Compute (nova) "in placement api an allocation reporter sometimes needs to be able to report an allocation even if it violates capacity constraints" [Undecided,New] - Assigned to Chris Dent (cdent) | 17:11 |
figleaf | cdent: I would log that differently, but remember that there is no GET for allocations | 17:11 |
figleaf | cdent: so sending different codes leaks information | 17:11 |
*** mingyu has joined #openstack-nova | 17:11 | |
cdent | I personally don't want to send a different code, but if superdan _needs_ it, then we need to talk about how to deal with that | 17:12 |
figleaf | cdent: huh, so we need both a PUT and a PUT_I_REALLY_MEAN_IT methods | 17:12 |
superdan | cdent: just a sec sorry | 17:13 |
cdent | figleaf: yeah, sort of. | 17:13 |
figleaf | cdent: I'll let you two fight it out. Got to step out for a few minutes | 17:13 |
cdent | enjoy | 17:13 |
*** trisq has quit IRC | 17:13 | |
*** tbachman has joined #openstack-nova | 17:13 | |
*** ttrumm has joined #openstack-nova | 17:15 | |
superdan | cdent: I don't think I need to know from the client side, no, I just need to know if what you have is what I have | 17:15 |
cdent | superdan: cool, i'll push what I've got up now (both the inventory and allocation stuff) and we can tweak it as required. and probably add some tests | 17:16 |
*** BjoernT is now known as Bjoern_zZzZzZzZ | 17:16 | |
*** mingyu has quit IRC | 17:16 | |
superdan | okay | 17:16 |
*** rmart04 has joined #openstack-nova | 17:17 | |
openstackgerrit | Chris Dent proposed openstack/nova: [placement] Allow inventory to violate allocations https://review.openstack.org/365068 | 17:18 |
openstackgerrit | Chris Dent proposed openstack/nova: [placement] don't write existing equivalent allocations https://review.openstack.org/365069 | 17:18 |
cdent | superdan: there go | 17:18 |
cdent | that's above the logging adjustments | 17:18 |
superdan | okay I think I just unfucked my local attempts to be graceful about inventories | 17:21 |
*** rmart04 has quit IRC | 17:21 | |
*** ttrumm has quit IRC | 17:22 | |
*** tongli has joined #openstack-nova | 17:25 | |
*** salv-orlando has quit IRC | 17:26 | |
*** tblakes has quit IRC | 17:28 | |
cdent | unfuckery is a positive | 17:28 |
*** markvoelker has joined #openstack-nova | 17:29 | |
mriedem | armax: how critical is it to get os-vif released with that LB fix? | 17:29 |
*** tongli has quit IRC | 17:29 | |
mriedem | like do i need to make a case to get a release exception with the release team before RC1? | 17:29 |
openstackgerrit | Dan Smith proposed openstack/nova: More conservative inventory updates https://review.openstack.org/365073 | 17:30 |
superdan | cdent: please review unfuckery ^ .. no unit test fixes for the changes yet | 17:30 |
*** mat128|away is now known as mat128 | 17:30 | |
mriedem | superdan: is that like what jaypipes had in his original one? | 17:32 |
mriedem | i remember his was a bit more elaborate on the client side checking | 17:32 |
*** paul-carlton has quit IRC | 17:32 | |
*** PaulMurray has quit IRC | 17:33 | |
*** markvoelker has quit IRC | 17:34 | |
*** ducttape_ has quit IRC | 17:34 | |
*** Bjoern_zZzZzZzZ is now known as BjoernT | 17:34 | |
* cdent looks | 17:34 | |
*** markvoelker has joined #openstack-nova | 17:34 | |
*** marst has quit IRC | 17:34 | |
superdan | mriedem: yeah, and this was my plan all along to follow up | 17:34 |
-openstackstatus- NOTICE: The infrastructure team is taking Gerrit offline for maintenance, beginning shortly after 18:00 UTC for a potentially 4 hour maintenance window. | 17:38 | |
* cdent gets superdan's code! | 17:39 | |
*** mriedem is now known as mriedem_afk | 17:39 | |
cdent | superdan: reading that it looks good, will make it run now | 17:39 |
openstackgerrit | viswesuwara nathan proposed openstack/nova: fixed flavor image listing if project_id is None https://review.openstack.org/363530 | 17:40 |
superdan | mriedem_afk: yeah sadly the server GET and PUT are different, so the process of knowing if we need to update them is... ugly | 17:42 |
superdan | it's almost 11 and I haven't had a morsel to eat, so ... gotta do that | 17:42 |
*** raj_singh_ has quit IRC | 17:42 | |
* superdan looks forward to fancy french marmalade | 17:42 | |
cdent | superdan: we should probably talk about changing that (GET and PUT, but also your food) next week | 17:42 |
cdent | it might be worth it | 17:42 |
openstackgerrit | Eric Brown proposed openstack/nova: Remove bandit.yaml in favor of defaults https://review.openstack.org/278544 | 17:43 |
cdent | allocations are being properly deleted when a server goes to ERROR | 17:44 |
superdan | cdent: yeah, we should talk to sean about it | 17:45 |
cdent | I think, next time we do something like this, that we should work in the other direction when both speccing and building. this time we worked database->api->things that will use it | 17:46 |
superdan | yeah, all the while saying we cared most that the api was good | 17:46 |
superdan | and alas... | 17:47 |
*** sudipto has quit IRC | 17:47 | |
cdent | i don't think we ever had a consistent and complete picture of ... I'm not even sure what word to use | 17:47 |
*** sudipto_ has quit IRC | 17:47 | |
cdent | oh well, it mostly works :) | 17:47 |
superdan | yeah | 17:47 |
superdan | I selfishly refused to review the api bits of placement out of spite, so I can't complain too much | 17:48 |
superdan | (not really spite, just being dramatic) | 17:48 |
*** lpetrut has quit IRC | 17:48 | |
*** tongli has joined #openstack-nova | 17:49 | |
cdent | it supports microversions, there's even a gist for how it will support microversion, we can change it when we need to | 17:49 |
superdan | I think we started our first microversion clock when we released it the first time, to perhaps we have a few days to course correct | 17:50 |
lascii | mriedem_afk: sorry, crazy day and I haven't been around my computer much. But yes we should have instance actions for swap volumes, and other things that are missing. Not worth waiting for tasks on doing that bit | 17:50 |
*** yamahata has joined #openstack-nova | 17:51 | |
*** sahid has quit IRC | 17:51 | |
*** sudipto has joined #openstack-nova | 17:53 | |
*** sudipto_ has joined #openstack-nova | 17:53 | |
cdent | I think I'm done for the day. If there's stuff that I can do email me, or assign a bug to me, or put something on a review, or whatever. | 17:53 |
*** tongli has quit IRC | 17:53 | |
* cdent waves | 17:53 | |
*** cdent has quit IRC | 17:54 | |
oomichi | mriedem_afk: about https://review.openstack.org/#/c/360941/1//COMMIT_MSG | 17:56 |
oomichi | mriedem_afk: I guess 404 is preferable I guess | 17:56 |
oomichi | mriedem_afk: but 400 is valid on this case based on https://github.com/openstack/api-wg/blob/master/guidelines/http.rst#failure-code-clarifications | 17:57 |
*** eharney_ has joined #openstack-nova | 17:57 | |
*** hemanthm is now known as hemanthm|lunch | 17:57 | |
*** eharney has quit IRC | 17:57 | |
*** ijw has joined #openstack-nova | 17:57 | |
*** rlrossit has quit IRC | 17:57 | |
*** eharney_ is now known as eharney | 17:58 | |
*** ijw has quit IRC | 17:59 | |
*** rlrossit has joined #openstack-nova | 17:59 | |
*** ijw has joined #openstack-nova | 18:01 | |
*** su_zhang has quit IRC | 18:01 | |
*** ducttape_ has joined #openstack-nova | 18:02 | |
*** rlrossit has quit IRC | 18:03 | |
*** ijw has quit IRC | 18:03 | |
*** annegentle has joined #openstack-nova | 18:04 | |
-openstackstatus- NOTICE: Gerrit is now going offline for maintenance, reserving a maintenance window through 22:00 UTC. | 18:05 | |
*** ChanServ changes topic to "Gerrit is now going offline for maintenance, reserving a maintenance window through 22:00 UTC." | 18:05 | |
*** su_zhang has joined #openstack-nova | 18:05 | |
*** su_zhang has quit IRC | 18:07 | |
*** rlrossit has joined #openstack-nova | 18:09 | |
*** rrecio_ has quit IRC | 18:09 | |
*** fragatina has quit IRC | 18:10 | |
*** _ducttape_ has joined #openstack-nova | 18:10 | |
armax | mriedem_afk: it’s biting all of us in the gate | 18:12 |
armax | mriedem_afk: the frequency is not too high so far but we’re in feature freeze | 18:12 |
armax | mriedem_afk: it’s not critical for prod deployments where one doesn’t run everything on a single node | 18:13 |
*** ducttape_ has quit IRC | 18:13 | |
armax | mriedem_afk: I am somewhat ambivalent if we can put up with the occasional gate reset | 18:13 |
*** mingyu has joined #openstack-nova | 18:14 | |
armax | mriedem_afk: until we can release in Ocata | 18:14 |
armax | kevinbenton: do you concur? ^ | 18:14 |
*** _ducttape_ has quit IRC | 18:14 | |
*** mnestratov has quit IRC | 18:16 | |
*** ijw has joined #openstack-nova | 18:16 | |
*** mingyu has quit IRC | 18:19 | |
*** ijw has quit IRC | 18:19 | |
*** sridharg has quit IRC | 18:20 | |
*** mamandle has quit IRC | 18:23 | |
*** TravT has quit IRC | 18:24 | |
*** dikonoo has quit IRC | 18:25 | |
*** sudipto has quit IRC | 18:25 | |
*** sudipto_ has quit IRC | 18:25 | |
*** aghadge has quit IRC | 18:28 | |
*** TravT has joined #openstack-nova | 18:28 | |
*** salv-orlando has joined #openstack-nova | 18:28 | |
*** su_zhang has joined #openstack-nova | 18:33 | |
*** ducttape_ has joined #openstack-nova | 18:35 | |
*** salv-orlando has quit IRC | 18:36 | |
*** tongli has joined #openstack-nova | 18:38 | |
*** ducttape_ has quit IRC | 18:39 | |
*** ducttape_ has joined #openstack-nova | 18:39 | |
*** aghadge has joined #openstack-nova | 18:41 | |
jroll | rpodolyaka: +1'd both, thanks for those :) | 18:41 |
*** lpetrut has joined #openstack-nova | 18:41 | |
*** tongli has quit IRC | 18:42 | |
*** cdent has joined #openstack-nova | 18:42 | |
rpodolyaka | jroll: thanks! | 18:43 |
*** aghadge has quit IRC | 18:43 | |
cdent | superdan: the icky format for get inventories is a bug: http://specs.openstack.org/openstack/nova-specs/specs/newton/approved/generic-resource-pools.html#get-resource-providers-uuid-inventories | 18:43 |
superdan | cdent: ah, yep | 18:43 |
superdan | good. | 18:43 |
superdan | :) | 18:43 |
cdent | yeah | 18:43 |
cdent | i'll make the bug and fix it sometime soon | 18:44 |
superdan | it's still different | 18:44 |
*** tblakes has joined #openstack-nova | 18:44 | |
superdan | but at least the generation is one per request | 18:44 |
superdan | the post takes a list of dicts with the key inside | 18:44 |
superdan | the get still returns a dict indexed by type | 18:44 |
superdan | but closer is better | 18:44 |
*** TravT has quit IRC | 18:44 | |
cdent | superdan: when I fix this I'll tune the resource tracker too | 18:48 |
superdan | cdent: okay.. looks like I need to fix some pep8 fails in your patches below | 18:48 |
superdan | you said you were done for the week, but .. here you are... anything pending/ | 18:48 |
*** TravT has joined #openstack-nova | 18:48 | |
cdent | yeah, I didn't do any of those kind of tests because I figured you want to play | 18:48 |
superdan | yep, it's fine | 18:49 |
cdent | yeah, I didn't mean to come back but my brain was tickling me that something wasn't right | 18:49 |
cdent | so I went looking.. | 18:49 |
superdan | okay | 18:49 |
*** rrecio_ has joined #openstack-nova | 18:49 | |
cdent | I don't plan to do anything tonight | 18:49 |
*** ducttape_ has quit IRC | 18:50 | |
*** fragatina has joined #openstack-nova | 18:51 | |
*** kfarr has quit IRC | 18:51 | |
*** ducttape_ has joined #openstack-nova | 18:52 | |
bauwser | cdent: so, about what we discussed today, do you have a change I could look at ? | 18:53 |
bauwser | cdent: or could I help you by implementing it ? | 18:54 |
cdent | bauwser: which thing? | 18:54 |
superdan | bauwser: gerrit is down anyway right? | 18:54 |
superdan | well, it's up now, but.. | 18:54 |
bauwser | cdent: the fact that we should just check if the allocation is existing and not check the inventory then | 18:55 |
superdan | yeah that is done | 18:55 |
bauwser | okay | 18:55 |
bauwser | I haven't seen the gerrit maintenance, sorry | 18:55 |
superdan | bauwser: here and down: https://review.openstack.org/#/c/365073/ | 18:55 |
superdan | bauwser: I feel like we should wait for jay to approve these since they are tweaks to the original plan | 18:56 |
superdan | but good to vet them in the meantime of course | 18:56 |
*** su_zhang has quit IRC | 18:57 | |
bauwser | fair point | 18:57 |
superdan | well, not my patch but the ones from cdent below | 18:57 |
*** su_zhang has joined #openstack-nova | 18:57 | |
bauwser | yup | 18:58 |
superdan | bauwser: the logging one at the bottom would be good to get in if you want to look at that and are itching to do something :D | 18:58 |
bauwser | I already reviewed it | 18:58 |
superdan | oh cool | 18:58 |
superdan | :) | 18:58 |
bauwser | but yeah | 18:58 |
superdan | FYI monday is a US holiday if you don't know | 18:59 |
*** su_zhang has quit IRC | 19:00 | |
*** raunak has quit IRC | 19:01 | |
*** ducttape_ has quit IRC | 19:02 | |
*** ducttape_ has joined #openstack-nova | 19:02 | |
*** sdake has quit IRC | 19:02 | |
bauwser | superdan: WAT ? you have some holiday that french don't ? I'm pissed off | 19:02 |
superdan | haha | 19:03 |
openstackgerrit | Dan Smith proposed openstack/nova: [placement] Allow inventory to violate allocations https://review.openstack.org/365068 | 19:03 |
openstackgerrit | Dan Smith proposed openstack/nova: [placement] don't write existing equivalent allocations https://review.openstack.org/365069 | 19:03 |
openstackgerrit | Dan Smith proposed openstack/nova: Move placement api request logging to middleware https://review.openstack.org/365015 | 19:03 |
openstackgerrit | Dan Smith proposed openstack/nova: More conservative inventory updates https://review.openstack.org/365073 | 19:03 |
superdan | pep8 fixes and tests for my changes ^ | 19:04 |
*** Jeffrey4l has quit IRC | 19:07 | |
*** cdent has quit IRC | 19:07 | |
*** tongli has joined #openstack-nova | 19:09 | |
*** tongli has quit IRC | 19:09 | |
*** tongli has joined #openstack-nova | 19:09 | |
*** tongli has quit IRC | 19:09 | |
*** tongli has joined #openstack-nova | 19:09 | |
*** ChanServ changes topic to "This channel is for Nova development. For support of Nova deployments, please use #openstack. Please see: https://wiki.openstack.org/wiki/Nova/Newton_Release_Schedule" | 19:14 | |
-openstackstatus- NOTICE: Gerrit is completing an online re-index, you may encounter slowness until it is complete | 19:14 | |
mriedem_afk | oomichi: i think you missed my point | 19:15 |
mriedem_afk | oomichi i wanted the DocImpact tag removed from the commit message | 19:16 |
*** TravT has quit IRC | 19:16 | |
mriedem_afk | ah i see it has been in the latest version | 19:16 |
openstackgerrit | MJWurtz proposed openstack/nova: Refresh info_cache after deleting floating IP https://review.openstack.org/357494 | 19:17 |
*** Nakato has quit IRC | 19:17 | |
*** Nakato has joined #openstack-nova | 19:19 | |
*** ducttape_ has quit IRC | 19:21 | |
*** nic has quit IRC | 19:21 | |
*** ducttape_ has joined #openstack-nova | 19:22 | |
*** delattec has joined #openstack-nova | 19:22 | |
*** ducttape_ has quit IRC | 19:24 | |
*** cdelatte has quit IRC | 19:25 | |
*** tbachman_ has joined #openstack-nova | 19:25 | |
*** tbachman has quit IRC | 19:25 | |
*** tbachman_ is now known as tbachman | 19:25 | |
*** delattec has quit IRC | 19:30 | |
*** bvanhav has quit IRC | 19:30 | |
*** ducttape_ has joined #openstack-nova | 19:31 | |
*** su_zhang has joined #openstack-nova | 19:31 | |
*** ducttape_ has quit IRC | 19:33 | |
*** whenry has quit IRC | 19:33 | |
*** gouthamr has quit IRC | 19:33 | |
*** thorst has quit IRC | 19:36 | |
*** thorst has joined #openstack-nova | 19:37 | |
*** su_zhang has quit IRC | 19:37 | |
*** markvoelker has quit IRC | 19:37 | |
*** thorst_ has joined #openstack-nova | 19:39 | |
*** thorst has quit IRC | 19:41 | |
*** links has joined #openstack-nova | 19:43 | |
*** thorst_ has quit IRC | 19:43 | |
*** rmcall has joined #openstack-nova | 19:44 | |
*** lpetrut has quit IRC | 19:45 | |
openstackgerrit | MJWurtz proposed openstack/nova: Refresh info_cache after deleting floating IP https://review.openstack.org/357494 | 19:45 |
*** jklare_ has quit IRC | 19:46 | |
*** jklare has joined #openstack-nova | 19:46 | |
*** su_zhang has joined #openstack-nova | 19:50 | |
*** annegentle has quit IRC | 19:51 | |
*** hemanthm|lunch is now known as hemanthm | 19:52 | |
*** links has quit IRC | 19:55 | |
*** pwnall has quit IRC | 19:55 | |
*** su_zhang has quit IRC | 19:56 | |
*** cjvolzka has joined #openstack-nova | 19:56 | |
*** mriedem_afk is now known as mriedem | 19:57 | |
mriedem | scottda: it doesn't seem that your retype test actually swaps the volume attachment on the server | 19:57 |
scottda | hmmm | 19:58 |
*** pwnall1 has joined #openstack-nova | 19:58 | |
scottda | mriedem: I'll have a look in a little while. | 20:00 |
*** TravT has joined #openstack-nova | 20:00 | |
mriedem | if retype doesn't create a new volume and swap to that, then i wouldn't expect it to call swap volume | 20:03 |
*** mingyu has joined #openstack-nova | 20:03 | |
oomichi | mriedem: Yeah, I see. and I just did catch the last part of your comment related to HTTP 400. | 20:03 |
*** akshai has quit IRC | 20:04 | |
*** openstackgerrit has quit IRC | 20:04 | |
*** openstackgerrit has joined #openstack-nova | 20:04 | |
*** TravT has quit IRC | 20:05 | |
*** armax has left #openstack-nova | 20:06 | |
*** armax has joined #openstack-nova | 20:06 | |
*** gouthamr has joined #openstack-nova | 20:07 | |
*** mingyu has quit IRC | 20:09 | |
cjvolzka | I see http://specs.openstack.org/openstack/nova-specs/specs/newton/approved/cells-instance-groups-api-db.html has been marked complete. The documentation says methods from db/sqalchemy/api.py would be mirrored to objects/instance_group.py. I don't see a replacement for instance_group_member_delete(). Is that still coming or was it dropped? | 20:09 |
*** su_zhang has joined #openstack-nova | 20:14 | |
melwitt | cjvolzka: the rest api doesn't allow delete of group members and the function instance_group_member_delete isn't called anywhere presently (it was never exposed by the instance group object interface) | 20:17 |
*** raunak has joined #openstack-nova | 20:19 | |
*** BjoernT has quit IRC | 20:20 | |
*** jamesdenton has quit IRC | 20:21 | |
mriedem | yeah you can't add or remove instance group members after the group is created right now | 20:21 |
mriedem | there was a spec at one point, it gets complicated | 20:21 |
cjvolzka | melwitt: It was a function that we happened to use internally. Was there already discussion about intentionally not including this any longer or was it just not added since there wasn't community use for it? In short I'm asking if it'd be worth while propose adding it back or has that already been decided. | 20:22 |
mriedem | cjvolzka: see https://review.openstack.org/#/q/owner:jmulsow%2540us.ibm.com+status:abandoned | 20:23 |
*** eharney has quit IRC | 20:25 | |
*** marst has joined #openstack-nova | 20:25 | |
*** ducttape_ has joined #openstack-nova | 20:25 | |
melwitt | cjvolzka: it wasn't discussed. the work was done to make instance group object functions work with both dbs and provide the migration and the delete members wasn't part of that since the object currently doesn't delete members as it is | 20:27 |
lascii | cjvolzka: I don't think there will be much support for adding it back if it's not exposed to a public interface | 20:28 |
*** wanghao_ has quit IRC | 20:30 | |
mriedem | we must fix this bug by RC1 | 20:31 |
mriedem | https://bugs.launchpad.net/nova/+bug/1618707 | 20:31 |
openstack | Launchpad bug 1618707 in OpenStack Compute (nova) "add comment" [Undecided,Invalid] - Assigned to wanghongtaozz (wanghongtaozz) | 20:31 |
melwitt | lol | 20:32 |
*** tblakes has quit IRC | 20:32 | |
*** tongli has quit IRC | 20:33 | |
*** gouthamr has quit IRC | 20:34 | |
harlowja_ | qq guys, mriedem melwitt (or others), wondering if a question we are debating on internally u guys can help me with | 20:35 |
harlowja_ | the question of notification messages out from nova (for say a instance) | 20:35 |
harlowja_ | is the ordering of the state changes guaranteed | 20:35 |
harlowja_ | (in cells this gets weird, but just let's exclude that for now) | 20:35 |
mriedem | is the ordering of the state changes gauranteed..... | 20:35 |
mriedem | ? | 20:35 |
harlowja_ | for the same instance | 20:35 |
harlowja_ | (at least) | 20:36 |
mriedem | have an example? | 20:36 |
harlowja_ | ya, though it involves cells, but its a ok example | 20:36 |
mriedem | gauranteed, i'd say no | 20:36 |
mriedem | since we have nothing testing that server actions go through a specific state transition | 20:36 |
mriedem | since we don't have a specific state machine | 20:36 |
harlowja_ | right, we have a instance.stop that we've seen come in before a instance.start (not often) | 20:36 |
mriedem | tdurakov is working on something like that, but more for migratoins | 20:36 |
harlowja_ | cells v1 i think is part of the bad part | 20:36 |
harlowja_ | bad == cause | 20:37 |
mriedem | is it a reboot operation? | 20:37 |
harlowja_ | unsure | 20:37 |
mriedem | if it's reboot, then that might make more sense | 20:37 |
harlowja_ | kk, jist of the thing is that i guess a specific state machine would help here (could at least be used to know the right order) | 20:37 |
mriedem | we talked a little bit about something like this at the newton midcycle, tdurakov was tasked with getting some of the state transitions for migrations down on paper as a start | 20:38 |
*** ducttape_ has quit IRC | 20:38 | |
mriedem | and then start working on a model, possibly based on what ironic is already doing | 20:38 |
harlowja_ | gotcha | 20:40 |
melwitt | state changes race in cells v1 because instance.save() causes a sync down to the compute cell. so those race with other actions going from parent to child cell. example https://bugs.launchpad.net/nova/+bug/1466696 | 20:41 |
openstack | Launchpad bug 1466696 in OpenStack Compute (nova) "Cells: Race between instance 'unlock' and 'stop' can cause 'stop' to fail" [Low,Won't fix] | 20:41 |
*** tonytan_brb has quit IRC | 20:42 | |
harlowja_ | ya the thing that we are theorizing is that we have a consumer on the child-cell that puts things into kafka and top-level consumer that also writes into kafka, the order sometimes gets busted in kafka (compounded by cells v1 i think) | 20:42 |
*** salv-orlando has joined #openstack-nova | 20:44 | |
harlowja_ | (not always gets busted) | 20:44 |
harlowja_ | just sometimes | 20:44 |
mriedem | tdurakov: fyi just saw this in bug triage https://bugs.launchpad.net/nova/+bug/1617299 | 20:45 |
openstack | Launchpad bug 1617299 in OpenStack Compute (nova) "Share based Nova Live Migration eratically fails" [Undecided,New] - Assigned to Tom Patzig (tom-patzig) | 20:45 |
*** thorst has joined #openstack-nova | 20:46 | |
melwitt | cells v1 would definitely do that sometimes when actions are done in rapid succession. I wouldn't expect it for non-cells but like mriedem it's really not guaranteed in any way as there's nothing formal about state transitions yet | 20:47 |
harlowja_ | melwitt right, think we've seen that :) | 20:47 |
*** markvoelker has joined #openstack-nova | 20:47 | |
harlowja_ | (the bug u mentioned) | 20:47 |
harlowja_ | btw how are u! | 20:47 |
harlowja_ | lol | 20:47 |
mriedem | you former yahoo! creeps need to tell those 2 that spam the operators list about no longer being at yahoo to knock it off | 20:48 |
harlowja_ | ya, i told | 20:48 |
mriedem | excellent | 20:48 |
harlowja_ | mriedem the message has been told | 20:48 |
harlowja_ | though i think its a larger issue with yahoo-inc email setup/server? | 20:48 |
harlowja_ | its not the first time this happens | 20:48 |
mriedem | tpatzig_: on https://bugs.launchpad.net/nova/+bug/1617299 can you put a retry backoff decorator in the code that checks for the test file on the other host during migration? | 20:48 |
openstack | Launchpad bug 1617299 in OpenStack Compute (nova) "Share based Nova Live Migration eratically fails" [Medium,Confirmed] - Assigned to Tom Patzig (tom-patzig) | 20:48 |
harlowja_ | there bouncing 'rule' needs adjustment :-P | 20:48 |
harlowja_ | stop replying to all when things bounce, lol | 20:49 |
*** erhudy has quit IRC | 20:52 | |
*** salv-orlando has quit IRC | 20:54 | |
*** gszasz has quit IRC | 20:56 | |
openstackgerrit | Ludovic Beliveau proposed openstack/nova: PCI: Update MAC address for NICs PF passthrough devices https://review.openstack.org/361438 | 20:56 |
*** browne has quit IRC | 21:00 | |
openstackgerrit | Matt Riedemann proposed openstack/nova: libvirt: improve logging for shared storage check https://review.openstack.org/365140 | 21:06 |
openstackgerrit | Matt Riedemann proposed openstack/nova: nfs fix for xenial images https://review.openstack.org/329466 | 21:07 |
*** eharney has joined #openstack-nova | 21:07 | |
*** liangy_ has quit IRC | 21:08 | |
*** salv-orlando has joined #openstack-nova | 21:08 | |
*** su_zhang has quit IRC | 21:11 | |
*** ayoung has quit IRC | 21:13 | |
*** su_zhang has joined #openstack-nova | 21:14 | |
mriedem | haleyb: unit tests are failing on https://review.openstack.org/#/c/313070/ | 21:16 |
*** ducttape_ has joined #openstack-nova | 21:16 | |
haleyb | mriedem: yes, i just haven't had time to fix it, wil try next week. With os-vif this isn't as important | 21:17 |
*** rtheis has quit IRC | 21:18 | |
*** sdake has joined #openstack-nova | 21:19 | |
*** ducttape_ has quit IRC | 21:20 | |
*** bauwser is now known as bauzas | 21:20 | |
*** ijw has joined #openstack-nova | 21:21 | |
*** lascii is now known as alaski | 21:27 | |
doffm | bauzas: So rlrossit has been doing some scheduler performance work... | 21:27 |
doffm | rlrossit: Want to take it away? | 21:27 |
rlrossit | let me get some pastes ready | 21:28 |
*** rmcall_ has joined #openstack-nova | 21:29 | |
rlrossit | bauzas: so I'm doing some performance testing on the scheduler, and I'm following the select_destinations notification to see how long it takes | 21:30 |
rlrossit | when I have debug logging turned on, select_destinations takes anywhere from 1-18 seconds, without debug logging, it takes 1-3 seconds | 21:30 |
*** rmcall has quit IRC | 21:30 | |
*** rmcall_ is now known as rmcall | 21:31 | |
rlrossit | this is in an environment where I have 400 faked compute nodes | 21:31 |
rlrossit | you can see the numbers here: http://paste.openstack.org/show/566153/ | 21:31 |
rlrossit | we determined that https://github.com/openstack/nova/blob/master/nova/scheduler/host_manager.py#L162-L178 is the main culprit | 21:31 |
rlrossit | because it's logging to debug 4 times for every host on every schedule when using the filter_scheduler | 21:31 |
rlrossit | so for me, that's 1600 debug log messages every time we run get_all_host_states() | 21:32 |
melwitt | debug logging isn't supposed to be turned on generally, but I get your point | 21:34 |
melwitt | bauzas might not be around right now, it's 23:34 in his time zone | 21:34 |
*** Guest55526 has quit IRC | 21:34 | |
rlrossit | I blame doffm | 21:34 |
*** intr1nsic has quit IRC | 21:35 | |
mriedem | rlrossit: talk to cfriesen | 21:36 |
mriedem | he's looked at a lot of the logging stuff in the scheduler | 21:36 |
*** browne has joined #openstack-nova | 21:36 | |
*** intr1nsic has joined #openstack-nova | 21:37 | |
melwitt | yeah, he's been working on a thing to only emit INFO log messages upon failure, or something similar. to avoid needing debug turned on to know anything | 21:38 |
doffm | Turning on debug right now makes the scheduler very very slow. | 21:38 |
melwitt | upon scheduling failure | 21:38 |
melwitt | yeah, that's known and no one should run with debug enabled unless they're actively troubleshooting a problem | 21:39 |
mriedem | doffm: rlrossit: http://specs.openstack.org/openstack/nova-specs/specs/newton/approved/improve-sched-logging.html | 21:39 |
mriedem | work on that for ocata | 21:39 |
doffm | Sounds good. | 21:40 |
*** ianw has quit IRC | 21:40 | |
*** cn28h has quit IRC | 21:41 | |
*** cn28h has joined #openstack-nova | 21:41 | |
*** tonytan4ever has joined #openstack-nova | 21:42 | |
*** ijw has quit IRC | 21:45 | |
*** hemanthm is now known as hemanthm|away | 21:45 | |
*** ducttape_ has joined #openstack-nova | 21:45 | |
*** rfolco has quit IRC | 21:45 | |
*** ducttape_ has quit IRC | 21:48 | |
*** tonytan4ever has quit IRC | 21:48 | |
*** ducttape_ has joined #openstack-nova | 21:48 | |
*** TravT has joined #openstack-nova | 21:51 | |
*** ducttape_ has quit IRC | 21:53 | |
*** ducttape_ has joined #openstack-nova | 21:54 | |
*** rrecio_ has quit IRC | 21:55 | |
*** esmiurium has quit IRC | 21:58 | |
*** ijw has joined #openstack-nova | 22:01 | |
*** esmiurium has joined #openstack-nova | 22:02 | |
*** ijw has quit IRC | 22:02 | |
*** rlrossit has quit IRC | 22:03 | |
auggy | i feel like there's a "taco trucks on every corner" pun to be made here but I'm struggling | 22:03 |
*** xyang1 has quit IRC | 22:09 | |
*** burt has quit IRC | 22:10 | |
oomichi | mriedem: do we have a chance to fix api-site in Newton? or should we wait for ocata now? | 22:10 |
oomichi | https://review.openstack.org/#/q/status:open+project:openstack/nova+branch:master+topic:bp/api-ref-in-rst | 22:10 |
oomichi | I am not sure that should be considered as high-prio *feature* or just fixing | 22:11 |
*** ianw has joined #openstack-nova | 22:11 | |
*** dimtruck is now known as zz_dimtruck | 22:12 | |
*** cn28h has quit IRC | 22:14 | |
*** cn28h has joined #openstack-nova | 22:14 | |
*** ijw has joined #openstack-nova | 22:15 | |
*** david-lyle has quit IRC | 22:15 | |
*** ijw has quit IRC | 22:15 | |
*** browne has quit IRC | 22:16 | |
*** ijw has joined #openstack-nova | 22:16 | |
*** david-lyle has joined #openstack-nova | 22:17 | |
*** pradk has quit IRC | 22:20 | |
*** su_zhang has quit IRC | 22:26 | |
*** snikitin_ has quit IRC | 22:28 | |
*** ijw has quit IRC | 22:32 | |
*** TravT has quit IRC | 22:36 | |
*** TravT has joined #openstack-nova | 22:36 | |
*** TravT has quit IRC | 22:36 | |
*** praneshp has quit IRC | 22:38 | |
*** claudiub has quit IRC | 22:38 | |
*** yamahata has quit IRC | 22:47 | |
*** markvoelker has quit IRC | 22:47 | |
*** ayoung has joined #openstack-nova | 22:48 | |
*** yamahata has joined #openstack-nova | 22:49 | |
*** karthiks has quit IRC | 22:50 | |
*** mmedvede has quit IRC | 22:51 | |
*** karthiks has joined #openstack-nova | 22:52 | |
*** mingyu has joined #openstack-nova | 22:53 | |
*** Apoorva has quit IRC | 22:53 | |
*** ijw has joined #openstack-nova | 22:55 | |
*** ijw has quit IRC | 22:56 | |
*** mingyu has quit IRC | 22:58 | |
*** ijw has joined #openstack-nova | 22:59 | |
*** jeblair is now known as corvus | 23:01 | |
*** hemna is now known as hemnafk | 23:05 | |
*** sameo_ has quit IRC | 23:11 | |
*** parora has quit IRC | 23:16 | |
*** mmedvede has joined #openstack-nova | 23:18 | |
*** ssurana has quit IRC | 23:18 | |
*** openstackgerrit has quit IRC | 23:18 | |
*** openstackgerrit has joined #openstack-nova | 23:19 | |
*** raunak has quit IRC | 23:21 | |
*** rama_y has quit IRC | 23:25 | |
*** sbezverk has joined #openstack-nova | 23:29 | |
*** thorst has quit IRC | 23:30 | |
*** rama_y has joined #openstack-nova | 23:31 | |
*** Swami has quit IRC | 23:32 | |
*** thorst has joined #openstack-nova | 23:32 | |
*** jamesdenton has joined #openstack-nova | 23:37 | |
*** ijw has quit IRC | 23:40 | |
*** ijw has joined #openstack-nova | 23:41 | |
*** artom has joined #openstack-nova | 23:42 | |
*** ijw has quit IRC | 23:45 | |
*** su_zhang has joined #openstack-nova | 23:48 | |
*** su_zhang has quit IRC | 23:48 | |
*** raunak has joined #openstack-nova | 23:49 | |
*** su_zhang has joined #openstack-nova | 23:49 | |
*** su_zhang has quit IRC | 23:53 | |
*** rama_y has quit IRC | 23:55 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!