| *** gvincent has joined #ara | 06:03 | |
| *** ktibi has joined #ara | 09:46 | |
| *** ktibi has quit IRC | 10:15 | |
| *** ktibi has joined #ara | 10:16 | |
| *** ktibi has quit IRC | 10:22 | |
| *** dbpiv has joined #ara | 11:23 | |
| *** dbpiv has quit IRC | 12:27 | |
| *** ktibi has joined #ara | 12:32 | |
| *** krion has quit IRC | 12:45 | |
| *** krion has joined #ara | 12:51 | |
| ktibi | dmsimard, Hi, So I succeded to install ARA api 1.0 (with PG) and configure ansible 2.8 with plugin. Do you have doc for install the web part or I need to use ansible role ? | 12:51 |
|---|---|---|
| dmsimard | ktibi: you can use the ansible role or install from source (which is ultimately not very different) | 12:53 |
| dmsimard | ktibi: https://github.com/ansible-community/ara-web -- npm install, npm start | 12:53 |
| dmsimard | the role takes care of installing npm, getting the source, generating either a static version or running the development server with a systemd unit | 12:53 |
| dmsimard | also takes care of the frontend (only nginx for now) | 12:53 |
| ktibi | dmsimard, ok thx for that. | 12:55 |
| ktibi | Do you have plan for multi tenant env for the web interface ? | 12:56 |
| ktibi | or I need to deploy a database and an API for each project ? (If I want to isolated each project) | 12:56 |
| dmsimard | ktibi: the web client connects to the API, so however many tenants you are sending to the api ? | 12:57 |
| dmsimard | there'll eventually be authentication support in the web client so it is able to authenticate against the API | 12:57 |
| ktibi | Maybe 10 project send to API. | 12:57 |
| dmsimard | you don't want a project to see another project's playbooks but yet still send them to the same database ? | 12:57 |
| ktibi | For now, I had imagined to deploy one API by project | 12:58 |
| ktibi | same PG instance but one database by project | 12:58 |
| dmsimard | that would work | 12:58 |
| dmsimard | I would need some time to think about what multi-tenancy would look like | 12:59 |
| dmsimard | very out of scope for 1.0 for sure :) | 13:00 |
| ktibi | for now, can I arrange all reports by label or project ? Maybe with a tag. | 13:00 |
| ktibi | If I use one API server | 13:00 |
| dmsimard | the API supports labelling playbooks | 13:00 |
| dmsimard | which, in turn, would allow you to search for playbooks matching a label | 13:01 |
| dmsimard | however there is no client-side support for that feature yet | 13:01 |
| ktibi | ok so I can't send the label with ansible. | 13:02 |
| dmsimard | well, not yet | 13:02 |
| dmsimard | hang on | 13:02 |
| dmsimard | ktibi: this is tracked as https://github.com/ansible-community/ara/issues/25 | 13:02 |
| ktibi | ho it's for that story board is empty | 13:03 |
| dmsimard | yeah, we migrated everything to github | 13:03 |
| dmsimard | labels might end up working the same way as playbook names, they aren't documented yet but are explained here https://github.com/ansible-community/ara/issues/15 | 13:03 |
| ktibi | ok, ara leave openstack community ? ;) | 13:03 |
| dmsimard | it hasn't | 13:03 |
| dmsimard | openstack has rebranded their code review/ci infrastructure to opendev.org and was no longer mirroring projects to the (now "official") openstack namespace on github | 13:04 |
| dmsimard | https://opendev.org/recordsansible/ara is the "authoritary" repository | 13:05 |
| dmsimard | github is the mirror | 13:05 |
| *** dbpiv has joined #ara | 13:08 | |
| *** ktibi has quit IRC | 13:15 | |
| *** ktibi has joined #ara | 13:16 | |
| *** ktibi has quit IRC | 13:17 | |
| *** ktibi has joined #ara | 13:18 | |
| ktibi | dmsimard, do you test ara with ansible 2.8 ? I have a lot of warning now : Failure using method (v2_playbook_on_stats) in callback plugin (<ansible.plugins.callback./home/ktibi/ansible3/lib/python3.6/site- | 13:22 |
| ktibi | packages/ara/plugins/callback/ara_default.CallbackModule object at 0x7fd48c2fd898>): 'NoneType' object is not subscriptable | 13:22 |
| dmsimard | ktibi: we have integration tests running on devel but it might be a regression we didn't catch | 13:22 |
| dmsimard | ktibi: does it always reproduce ? | 13:22 |
| ktibi | dmsimard, https://paste.ubuntu.com/p/TMNxSPKmvr/ | 13:24 |
| dmsimard | ktibi: can you run that with -vvv (three V's) to have the full trace ? | 13:25 |
| ktibi | dmsimard, I pasted just full trace, I removed all ansible log : https://paste.ubuntu.com/p/MHbttQfRP7/ | 13:28 |
| dmsimard | ok, give me a while and I can look into it | 13:31 |
| dmsimard | apollo13: I was looking at the client side implementation for labels and asking myself whether we really need a dedicated table just for holding the labels | 13:45 |
| dmsimard | could probably just be a field in the playbook table ? | 13:45 |
| dmsimard | the objective is to make labels searchable | 13:46 |
| dmsimard | i.e, finding playbooks that have label X or Y | 13:46 |
| dmsimard | I figure it's probably a faster query to look up playbooks that match a label which under the hood is a foreign key ? | 13:48 |
| dmsimard | the only thing with a dedicated table is that we need to handle the machinery of creating the label if it doesn't exist | 13:48 |
| dmsimard | so need to change https://github.com/ansible-community/ara/blob/e1d3e568937a50916b468c7a1713fa092eb3fc2e/ara/api/serializers.py#L366-L375 slightly I suppose | 13:52 |
| dmsimard | yeah I'll stick with a dedicated table -- otherwise if you change the name of a label it doesn't cascade | 14:11 |
| apollo13 | dmsimard: also if it is just a field, how are you going to search if you have multiple labels | 14:32 |
| apollo13 | aside from an array field or so which basically only work on postgresql :D | 14:32 |
| dmsimard | I actually sort of wish jsonfield worked with sqlite | 14:33 |
| dmsimard | and mysql | 14:33 |
| dmsimard | since ansible is all json | 14:33 |
| apollo13 | it more or less does | 14:33 |
| apollo13 | well differently put, it would be possible to write a json field for those three | 14:34 |
| apollo13 | I think django has a gsoc project this year trying to do that | 14:34 |
| dmsimard | the serialized and compressed binary fields we have works right now but there are surely opportunities to improve that | 14:34 |
| dmsimard | ktibi: oi, I might have a patch for labels sometime today if you want to try it out | 14:36 |
| dmsimard | ^ apollo13: I think you wanted to try that as well | 14:36 |
| apollo13 | I'll do my best, no promises though | 14:37 |
| apollo13 | I still have to read the docs and figure out how to install the webfrontend :D | 14:37 |
| dmsimard | npm install/npm start :p | 14:37 |
| dmsimard | otherwise https://ara.readthedocs.io/en/feature-1.0/ansible-role-ara-web.html | 14:38 |
| apollo13 | mhm, the frontend should just be static files right? | 14:38 |
| dmsimard | it can be, yes | 14:38 |
| *** ktibi has quit IRC | 14:38 | |
| dmsimard | # Whether to use the embedded react web server or not | 14:39 |
| dmsimard | # Setting this to false means ara-web will be statically built instead | 14:39 |
| dmsimard | ara_web_dev_server: true | 14:39 |
| *** krion has quit IRC | 14:41 | |
| apollo13 | cool, so that I could include into my exe then, let me quickly check if I can built an executable for ara-server :D | 14:41 |
| dmsimard | hmmmm | 14:41 |
| dmsimard | for labels | 14:41 |
| *** krion has joined #ara | 14:42 | |
| dmsimard | I want users to just pass a list of labels | 14:42 |
| dmsimard | but the API is expecting a list of dicts (which would be [{"name": "foo"}]) instead of just ["foo"] | 14:42 |
| apollo13 | map(lambda l: {"name": l}, labels) | 14:43 |
| apollo13 | will give you that | 14:43 |
| dmsimard | sure, but I was debating where that code should live | 14:43 |
| dmsimard | in the callback would work | 14:44 |
| dmsimard | but then you need to patch all the other places too | 14:44 |
| dmsimard | so I guess it needs to be at the serializer level | 14:44 |
| apollo13 | right, change the api? :D | 14:44 |
| *** ktibi has joined #ara | 14:50 | |
| apollo13 | mhm, my idea of building ara into an executable fails due to pyyaml not having manylinux wheels *scratches head and wonders if he really wants to go down that rabbit hole* | 14:51 |
| apollo13 | otherwise the executable seems to run, guess I have to try it in a manylinux 2010 container and see if I can build it | 14:52 |
| apollo13 | dmsimard: are you up for an adventure? | 15:05 |
| dmsimard | oh boy | 15:05 |
| dmsimard | we're friday, don't ask too much of me | 15:06 |
| apollo13 | if you trust me, download https://apolloner.eu/~apollo13/.tmp/ara | 15:06 |
| apollo13 | make it executable and see if you can get it to run | 15:06 |
| dmsimard | apollo13: that seems to work | 15:07 |
| apollo13 | coolio | 15:07 |
| dmsimard | don't think it's the latest tag though, has the cors bug | 15:07 |
| apollo13 | that is a standalone ara server | 15:07 |
| apollo13 | it is the beta build yes | 15:07 |
| apollo13 | from pypi | 15:07 |
| dmsimard | hmm | 15:07 |
| dmsimard | ?: (corsheaders.E013) Origin '127.0.0.1:8000' in CORS_ORIGIN_WHITELIST is missing scheme or netloc | 15:07 |
| dmsimard | ?: (corsheaders.E013) Origin 'localhost:3000' in CORS_ORIGIN_WHITELIST is missing scheme or netloc | 15:07 |
| dmsimard | that's supposed to be fixed by the new defaults which include the http:// | 15:07 |
| apollo13 | ara-1.0.0.0b1-py2.py3-none-any.whl | 15:08 |
| apollo13 | straight from pypi, so apparently your config is still wonky or it's not fixed :D | 15:08 |
| dmsimard | oh, wait, that's possible | 15:08 |
| dmsimard | yeah | 15:08 |
| dmsimard | old config | 15:08 |
| dmsimard | ++ it works | 15:08 |
| dmsimard | how did you do it ? | 15:09 |
| apollo13 | now all I need to do is add gunicorn + the static files from the frontend and we got a server running in a second | 15:09 |
| dmsimard | yeah I think the api is a good fit for containers | 15:09 |
| apollo13 | pex -m ara.server PyYAML-5.1-cp37-cp37m-manylinux1_x86_64.whl 'ara[server]==1.0.0.0b1' psycopg2-binary -o ara | 15:09 |
| dmsimard | that's it ? | 15:10 |
| apollo13 | yeah | 15:10 |
| dmsimard | that's pretty friggin sweet | 15:10 |
| apollo13 | well I did manually build PyYAML in a manylinux container otherwise it would build against my system and most likely not work on yours | 15:10 |
| apollo13 | but aside from that | 15:10 |
| dmsimard | that's because the wheels aren't on pypi ? | 15:10 |
| apollo13 | yeah | 15:10 |
| dmsimard | https://github.com/yaml/pyyaml/issues/95 | 15:11 |
| apollo13 | if I add gunicorn and a custom management command, you can even run "ara server" and do socket activation via systemd (at least if a gunicorn with my patches is used) | 15:11 |
| apollo13 | yeah https://github.com/yaml/pyyaml/issues/43 mainly | 15:12 |
| dmsimard | gvincent: around ? scratching my head about a django-rest-framework serializer issue and google-fu is failing me | 15:15 |
| gvincent | yes | 15:15 |
| dmsimard | gvincent: it's about playbooks and labels -- somewhere along those lines: https://github.com/ansible-community/ara/blob/e1d3e568937a50916b468c7a1713fa092eb3fc2e/ara/api/serializers.py#L348-L375 | 15:16 |
| dmsimard | 1) I got rid of the description field which was redundant (people can just supply good names I suppose) | 15:17 |
| dmsimard | 2) I'd like the client to be able to pass a list of label names: ["foo", "bar"] instead of needing to supply dicts: [{"name": "foo"}, {"name", "bar"}] | 15:18 |
| dmsimard | 3) I already changed the models.Label.objects.create to a models.Label.objects.get_or_create | 15:18 |
| dmsimard | The label serializer is preventing me from passing just a string that would be the name, it really wants a dict with the name key | 15:19 |
| dmsimard | any ideas ? | 15:19 |
| gvincent | 2sec | 15:20 |
| dmsimard | gvincent: it seems like this is close to what I'm looking for https://stackoverflow.com/questions/40060332/django-rest-framework-default-serializer-for-foreign-key-field | 15:30 |
| dmsimard | the to_representation part | 15:31 |
| gvincent | re | 15:34 |
| gvincent | dmsimard, I don't understand The label serializer is preventing me from passing just a string | 15:36 |
| gvincent | for which field? | 15:36 |
| dmsimard | gvincent: http://paste.openstack.org/raw/751525/ | 15:36 |
| gvincent | dmsimard, you need to overwrite create on playbook serializer | 15:38 |
| dmsimard | I already am but I think it's the LabelSerializer in PlaybookSerializer that is causing the issue | 15:38 |
| dmsimard | if I remove the LabelSerializer, it changes the error to "Incorrect type. Expected pk value, received str." | 15:39 |
| gvincent | dmsimard, | 15:40 |
| gvincent | https://www.django-rest-framework.org/api-guide/relations/#slugrelatedfield | 15:40 |
| dmsimard | oh, let me try that. | 15:40 |
| gvincent | labels = serializers.SlugRelatedField( | 15:43 |
| gvincent | many=True, | 15:43 |
| gvincent | queryset=Label.objects.all(), | 15:43 |
| gvincent | slug_field='text' | 15:43 |
| gvincent | ) | 15:43 |
| gvincent | or something like that | 15:43 |
| gvincent | I need to go | 15:43 |
| gvincent | feel free to put a -1 review with the test red, I can have a look on monday | 15:44 |
| dmsimard | gvincent: np, I'll do some work with that. It sort of works but we need to create the label if it doesn't exist | 15:44 |
| dmsimard | i.e, "Object with name=test-label does not exist." | 15:44 |
| dmsimard | but getting closer | 15:44 |
| gvincent | dmsimard, so read_only=True, | 15:44 |
| *** dbpiv has quit IRC | 15:46 | |
| dmsimard | it works \o/ | 16:10 |
| dmsimard | gvincent: <3 thanks | 16:10 |
| gvincent | your welcome | 16:10 |
| gvincent | you're* | 16:11 |
| dmsimard | it is super clean too | 16:11 |
| dmsimard | no need to override the create method | 16:11 |
| gvincent | with read_only=True? | 16:11 |
| gvincent | label are not created? | 16:12 |
| dmsimard | gvincent: need a slugrelatedfield that supports get_or_create, e.g https://stackoverflow.com/a/28011896 | 16:13 |
| gvincent | ah yes ok | 16:13 |
| gvincent | I understand | 16:13 |
| gvincent | dmsimard, I will implement https://github.com/ansible-community/ara-web/issues/7 | 16:15 |
| dmsimard | sure | 16:16 |
| dmsimard | I wanted to give a shot at implementing the file/host facts modals | 16:16 |
| gvincent | It's a good idea to regroup everythink by task | 16:16 |
| dmsimard | but I lost patience trying to figure it out haha | 16:16 |
| gvincent | I think I can do it locally without modifying the API | 16:17 |
| gvincent | haha ;) | 16:17 |
| gvincent | I'm in my new house now, I have more time to work on ara | 16:17 |
| gvincent | sorry for my absence | 16:17 |
| dmsimard | gvincent: the problem with that is that the list of results is only available in the detailed task view | 16:17 |
| dmsimard | i.e, https://api.demo.recordsansible.org/api/v1/tasks doesn't provide the list of results | 16:17 |
| dmsimard | however https://api.demo.recordsansible.org/api/v1/results provides their parent task | 16:18 |
| dmsimard | anyway, let me know if you have questions | 16:18 |
| dmsimard | happy weekend :D | 16:18 |
| *** ktibi has quit IRC | 16:19 | |
| openstackgerrit | David Moreau Simard proposed recordsansible/ara feature/1.0: Remove the description field from the label model https://review.opendev.org/659834 | 16:46 |
| openstackgerrit | David Moreau Simard proposed recordsansible/ara feature/1.0: Add support for specifying playbook labels as Ansible variables https://review.opendev.org/659835 | 16:46 |
| openstackgerrit | David Moreau Simard proposed recordsansible/ara feature/1.0: Add missing test for patching a playbook name https://review.opendev.org/659836 | 16:47 |
| openstackgerrit | David Moreau Simard proposed recordsansible/ara feature/1.0: Add missing test for patching a playbook name https://review.opendev.org/659836 | 16:48 |
| openstackgerrit | David Moreau Simard proposed recordsansible/ara feature/1.0: Remove the description field from the label model https://review.opendev.org/659834 | 17:20 |
| openstackgerrit | Merged recordsansible/ara feature/1.0: Add missing test for patching a playbook name https://review.opendev.org/659836 | 17:33 |
| openstackgerrit | Merged recordsansible/ara feature/1.0: Remove the description field from the label model https://review.opendev.org/659834 | 17:47 |
| apollo13 | dmsimard: does zuul support running docker containers? if yes we can probably relatively easily build an ara exectuable | 17:49 |
| dmsimard | apollo13: zuul executes arbitrary code on VMs, it can do whatever you throw at it | 17:50 |
| dmsimard | for the postgresql integration tests, the job installs docker, spins up a postgre container and uses it | 17:50 |
| dmsimard | it's a bit finicky because the ansible that zuul provides doesn't have the docker python lib, for example | 17:50 |
| dmsimard | so you can run docker shell commands but not the docker ansible modules | 17:51 |
| apollo13 | hehe, so if I write a bash script that performs the building you will do the CI for me? :) | 17:51 |
| dmsimard | apollo13: https://github.com/ansible-community/ara/blob/d04d5d21fe02d7d612a9a1efaee2ae7acbbbe392/tests/with_postgresql.yaml#L34-L62 | 17:51 |
| apollo13 | actually I probably can write the ansible stuff too, might just need help with zuul oddities | 17:51 |
| dmsimard | apollo13: start with the thing that builds your thing | 17:51 |
| apollo13 | yeah | 17:52 |
| dmsimard | and then I can point you in the right direction | 17:52 |
| apollo13 | will give it a shot over the next days | 17:52 |
| dmsimard | ++ | 17:52 |
| apollo13 | having an executable + aa systemd service fail as only requirement for ara server seems a nice achievement | 17:52 |
| dmsimard | FWIW I've been meaning to try https://github.com/ansible-community/ansible-bender which could supposedly build us container images straight from our existing roles and playbooks | 17:53 |
| apollo13 | interesting | 17:53 |
| apollo13 | so much to do, so much to try :D | 17:53 |
| dmsimard | last time I tried it I think I ran into this bug https://github.com/ansible/ansible/pull/55816 | 17:53 |
| dmsimard | well, that's the PR of the bug | 17:54 |
| dmsimard | larsks: ^ thanks for that btw, need to try it :p | 17:54 |
| apollo13 | ok, onto the bus, cy… | 17:55 |
| dmsimard | apollo13: yeah, it never occurred to me to make binary blobs | 17:55 |
| dmsimard | It's not like we're writing golang or anything :p | 17:55 |
| dmsimard | have a nice weekend o/ | 17:55 |
| *** gvincent has quit IRC | 20:08 | |
| dmsimard | I put a database from https://review.opendev.org/#/c/659835/ up on the live demo, we now have labels https://api.demo.recordsansible.org/api/v1/labels | 22:14 |
| dmsimard | I need to figure out how to make the label field searchable | 22:20 |
| dmsimard | found it \o/ | 22:30 |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!