opendevreview | Ghanshyam proposed openstack/nova master: HyperV: Remove RDP console API https://review.opendev.org/c/openstack/nova/+/906809 | 06:43 |
---|---|---|
opendevreview | Ghanshyam proposed openstack/nova master: HyperV: Remove RDP console API https://review.opendev.org/c/openstack/nova/+/906809 | 06:44 |
opendevreview | zhouzhong proposed openstack/nova master: Validate flavor image min ram when resizing a volume-backend instance https://review.opendev.org/c/openstack/nova/+/906813 | 07:27 |
TK_ | Hello Team, I have an openstack environment with 4 compute Nodes however yesterday out of the blue, the Instances on one of the compute nodes started shutting themselves down. Then I realized the the compute service was disabled when I checked in Horizon. I enabled it and restarted the instances and all was well but I cant figure out what caused it. I have attached the nova_compute logs here incase someone would like to support | 08:08 |
TK_ | https://paste.openstack.org/show/bxqpgjvqT3DlOYWoeejc/ | 08:08 |
opendevreview | melanie witt proposed openstack/nova master: libvirt: Configure and teardown ephemeral encryption secrets https://review.opendev.org/c/openstack/nova/+/826754 | 08:12 |
opendevreview | melanie witt proposed openstack/nova master: imagebackend: Add support to libvirt_info for LUKS based encryption https://review.opendev.org/c/openstack/nova/+/826755 | 08:12 |
opendevreview | melanie witt proposed openstack/nova master: Add encryption support to convert_image https://review.opendev.org/c/openstack/nova/+/870934 | 08:12 |
opendevreview | melanie witt proposed openstack/nova master: Support create with ephemeral encryption for qcow2 https://review.opendev.org/c/openstack/nova/+/870932 | 08:12 |
opendevreview | melanie witt proposed openstack/nova master: Support resize with ephemeral encryption for qcow2 https://review.opendev.org/c/openstack/nova/+/870933 | 08:12 |
opendevreview | melanie witt proposed openstack/nova master: Add hw_ephemeral_encryption_secret_uuid image property https://review.opendev.org/c/openstack/nova/+/870935 | 08:12 |
opendevreview | melanie witt proposed openstack/nova master: Add encryption support to qemu-img rebase https://review.opendev.org/c/openstack/nova/+/870936 | 08:12 |
opendevreview | melanie witt proposed openstack/nova master: Support snapshot with ephemeral encryption for qcow2 https://review.opendev.org/c/openstack/nova/+/870937 | 08:12 |
opendevreview | melanie witt proposed openstack/nova master: Support rebuild and unshelve with ephemeral encryption https://review.opendev.org/c/openstack/nova/+/870939 | 08:12 |
opendevreview | melanie witt proposed openstack/nova master: Support rescue with ephemeral encryption https://review.opendev.org/c/openstack/nova/+/873675 | 08:12 |
opendevreview | melanie witt proposed openstack/nova master: libvirt: make <encryption> a sub element of <source> https://review.opendev.org/c/openstack/nova/+/905515 | 08:12 |
opendevreview | melanie witt proposed openstack/nova master: WIP Support migration with ephemeral encryption https://review.opendev.org/c/openstack/nova/+/905512 | 08:12 |
opendevreview | melanie witt proposed openstack/nova master: Reject (resize|rebuild) API requests with conflicting encryption https://review.opendev.org/c/openstack/nova/+/904240 | 08:12 |
opendevreview | melanie witt proposed openstack/nova master: libvirt: Introduce support for qcow2 with LUKS https://review.opendev.org/c/openstack/nova/+/772273 | 08:12 |
opendevreview | melanie witt proposed openstack/nova master: libvirt: Introduce support for raw with LUKS https://review.opendev.org/c/openstack/nova/+/884313 | 08:12 |
opendevreview | melanie witt proposed openstack/nova master: libvirt: Introduce support for rbd with LUKS https://review.opendev.org/c/openstack/nova/+/889912 | 08:12 |
opendevreview | melanie witt proposed openstack/nova master: DNM test ephemeral encryption + resize: qcow2, raw, rbd https://review.opendev.org/c/openstack/nova/+/862416 | 08:16 |
bauzas | TK_: we already replied here to your problem, this is due to some evacuations. Please also look at the channel topic, please | 08:33 |
tkajinam | bauzas, looks like he's been repeatedly asking the same question in multiple places (irc and ml). idk what's going wrong | 09:39 |
* tkajinam shrugs | 09:40 | |
* tkajinam runs away | 09:40 | |
*** ravlew is now known as Guest444 | 10:09 | |
gibi | Uggla: I probably asked this before, sorry I lost most of the context. Do you have a devstack local.conf that has the necessary configuration to play with your manila series locally? | 10:30 |
Uggla | @gibi, yes I have one for ceph and one for nfs | 10:31 |
gibi | Uggla: can I have them? :) | 10:31 |
Uggla | sure | 10:31 |
opendevreview | Sylvain Bauza proposed openstack/nova master: Reserve mdevs to return to the source https://review.opendev.org/c/openstack/nova/+/904209 | 10:46 |
opendevreview | Sylvain Bauza proposed openstack/nova master: Modify the mdevs in the migrate XML https://review.opendev.org/c/openstack/nova/+/904258 | 10:46 |
bauzas | fwiw, guest kernel panics are back https://ccd793ab1dcad57d4bce-30f7f95167373513d0da56e7a7846412.ssl.cf1.rackcdn.com/904209/10/check/tempest-integrated-compute/1981c65/testr_results.html | 14:59 |
bauzas | yay... | 14:59 |
bauzas | /sbin/init: can't load library 'libtirpc.so.3' | 15:00 |
bauzas | something looks wrong to me with some I/O | 15:00 |
bauzas | I wish we could have something like dstat or vmstat in https://ccd793ab1dcad57d4bce-30f7f95167373513d0da56e7a7846412.ssl.cf1.rackcdn.com/904209/10/check/tempest-integrated-compute/1981c65/controller/logs/index.html | 15:05 |
opendevreview | Sahid Orentino Ferdjaoui proposed openstack/nova master: scheduler: AggregateMultitenancyIsolation to support unlimited tenant https://review.opendev.org/c/openstack/nova/+/896512 | 15:06 |
lowercase | Morning, nova-manage db sync is reporting that it is not able to upgrade nova to 26.4.0 (zed), from yoga. The error is, unknown revision 960aac0e09ea | 15:11 |
lowercase | I went back to yoga, performed a db sync again and it reports the same error | 15:11 |
kevko | lowercase: show your log via paste | 15:12 |
lowercase | One moment, corporate firewall hates pastebins so let me find one that is unblocked | 15:12 |
tkajinam | lowercase, maybe you can try https://paste.opendev.org/ ? | 15:16 |
tkajinam | anyone from the core can check this stale one and abandon it ? https://review.opendev.org/c/openstack/nova/+/677870 | 15:16 |
lowercase | kevko: https://paste.centos.org/view/d9542667 | 15:17 |
lowercase | tkajinam: that one does work, thank you. | 15:17 |
kevko | lowercase: can u access database ? | 15:18 |
lowercase | yes | 15:18 |
lowercase | I am logged into the db's now | 15:19 |
kevko | lowercase: and what about select * from cell_mappings; in nova_api database ? do you have right creds inside ? | 15:20 |
gibi | tkajinam: done | 15:20 |
tkajinam | gibi, thx ! | 15:20 |
lowercase | kevko: nova, nova_api nova_cell0 database? | 15:20 |
kevko | in nova_api database i said | 15:20 |
lowercase | sorry | 15:20 |
kevko | and what about api_db sync ? is it same error ? | 15:21 |
kevko | do you have right credentials in /etc/nova/nova.conf ? | 15:21 |
lowercase | This cluster is old btw.. old old | 15:21 |
lowercase | might have been created around juno or kilo or so | 15:22 |
lowercase | so I personally created the cell0 creds | 15:22 |
kevko | and you want to upgrade now ? :D :D :D | 15:22 |
lowercase | sorry, last statement sounded a bit ego. | 15:22 |
lowercase | what i meant to say, is that i know these creds are correct is because i was the one that upgraded the nova database when cell0 was introduced | 15:23 |
kevko | sorry, but i think you need to go from version to version | 15:23 |
kevko | you cannot just run db migrations from such new version | 15:23 |
lowercase | yes, this openstack environment has never skipped versions | 15:23 |
lowercase | the entire openstack infra has gone 1 major release at a time all the way to yoga | 15:23 |
kevko | hmm | 15:23 |
lowercase | that's why I was digging through the code to discover which arambic version is associated with 960aac0e09ea | 15:25 |
kevko | sorry but i think i can't help here | 15:27 |
lowercase | Np, last time i had this issue I had to step through the code manually to get it to upgrade. | 15:28 |
lowercase | It wasn't this environment... | 15:28 |
lowercase | or was it? | 15:28 |
kevko | lowercase: but i still don't udenrstand why it's saying that can't connect to DB | 15:28 |
kevko | do you see mysql attempt ? | 15:28 |
lowercase | idk, I don't see Moody online | 15:28 |
kevko | what about your nova.conf ..can u share ? | 15:29 |
lowercase | Oh, you are looking that the lines from the db manage command that ask to ensure the db connection is good? | 15:30 |
lowercase | yeah sorry, that's a strong red herring | 15:30 |
lowercase | So each version of openstack is associated with a structure or schema of the database. The database scheme changes based on the needs of the devs and what information needs to be stored. To ensure the code knows which version the database is on, they use a database version number. In this case, "960aac0e09ea" denotes the last version the nova-api database is on. Zed's | 15:32 |
lowercase | nova-manage db sync, goes in, inspects the version number. And returns an error stating that it doesn't know which SCHEME the database is on and thus cannot upgrade it. | 15:32 |
lowercase | What normally happens is nova-manage db sync looks at the previous version number, modifies the schema of the database, and replaces the arambic (sp?) version number to be the new database schema. | 15:33 |
lowercase | I think this is what I'm looking for: https://github.com/openstack/nova/tree/master/nova/db/api/migrations/versions | 15:38 |
lowercase | At the top of the code it mentions: Revision ID: cdeec0c85668 Is the version of the database schema is GOING TO, and the Revises: b30f573d3377 says what the database schema is COMING FROM | 15:39 |
lowercase | i swear to god, kevko if your a dev and you're just letting me ramble on while you know this shit and didn't tell me I will feel like a fool. | 15:40 |
kevko | lowercase: I am ops .. | 16:14 |
kevko | lowercase: didn't know ..i wanted to just help .. | 16:14 |
kevko | or devops | 16:14 |
opendevreview | sean mooney proposed openstack/nova master: [WIP] add iniital health check to compute agent https://review.opendev.org/c/openstack/nova/+/906911 | 18:44 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!