'NoneType' object has no attribute 'append_message' and other Debian (post-setup) trouble


#21

Anything the sudo user configured on the worker pool can write to.

/opt/vespene_build maybe


#22

Edit: I just realized I was writing this message for over an hour and trying out things in between. tldr; I made it work but I don't know how to automate it in a setup script. There might be a different way though.

I added the djnd (sudo user) to the vespene group and chgrp'd and chmod'd the directory to vespene and 770. This is the log right now, it comes further than last time:

2018-11-02 18:46:50,044 - vespene - DEBUG - building: 55, project: stanovanja-build
2018-11-02 18:46:50,107 - vespene - DEBUG - executing: mkdir -p /opt/vespene_build/55
2018-11-02 18:46:50,130 - vespene - DEBUG - executing: id
2018-11-02 18:46:50,182 - vespene - DEBUG - uid=1001(vespene) gid=1001(vespene) groups=1001(vespene)

2018-11-02 18:46:50,182 - vespene - DEBUG - executing: chmod 770 /opt/vespene_build/55
2018-11-02 18:46:50,231 - vespene - DEBUG - adding SSH key with passphrase!
2018-11-02 18:46:50,233 - vespene - DEBUG - /tmp/tmpz249wde_
2018-11-02 18:46:50,233 - vespene - DEBUG - /tmp/tmpucr18ql7
2018-11-02 18:46:50,233 - vespene - DEBUG - executing: /usr/bin/expect -f /tmp/tmpz249wde_
2018-11-02 18:46:50,327 - vespene - DEBUG - executing: timeout 600 git clone -b master --single-branch git@github.com:danesjenovdan/stanovanja.git /opt/vespene_build/55 -v
2018-11-02 18:46:51,970 - vespene - DEBUG - Warning: Permanently added 'github.com,192.30.253.113' (RSA) to the list of known hosts.

2018-11-02 18:46:51,970 - vespene - DEBUG - executing: (cd /opt/vespene_build/55; git --no-pager log -n 1 --abbrev-commit --format='%aE')
2018-11-02 18:46:52,019 - vespene - DEBUG - executing: (cd /opt/vespene_build/55; git rev-parse --short HEAD)
2018-11-02 18:46:52,103 - vespene - DEBUG - executing: id
2018-11-02 18:46:52,112 - vespene - DEBUG - uid=1001(vespene) gid=1001(vespene) groups=1001(vespene)

2018-11-02 18:46:52,113 - vespene - DEBUG - executing: chmod -R 777 /opt/vespene_build/55
2018-11-02 18:46:52,139 - vespene - DEBUG - executing: chmod a+x /opt/vespene_build/55/vespene_launch.sh
2018-11-02 18:46:52,248 - vespene - ERROR - an error occurred
Traceback (most recent call last):
  File "/opt/vespene/vespene/workers/builder.py", line 160, in go
    self.main()
  File "/opt/vespene/vespene/workers/builder.py", line 69, in main
    self.run_build_script()
  File "/opt/vespene/vespene/workers/builder.py", line 91, in run_build_script
    self.isolation_manager.execute()
  File "/opt/vespene/vespene/workers/isolation.py", line 55, in execute
    return self.provider.execute()
  File "/opt/vespene/vespene/plugins/isolation/sudo.py", line 97, in execute
    commands.execute_command(self.build, sudo_command, log_command=False, output_log=True, message_log=False)
  File "/opt/vespene/vespene/workers/commands.py", line 132, in execute_command
    raise Exception("Failed")
Exception: Failed
2018-11-02 18:46:52,250 - vespene - DEBUG - flagging build as failure
2018-11-02 18:46:52,285 - vespene - DEBUG - flagging build as done

The Output in the UI says: sudo: a password is required. I logged the self.sudo_password value and it is correct. Interestingly, if I run the sudo_command by hand as the djnd user it runs the script I entered into the UI.

If I run the command echo "passpasssomepass" | sudo -Snk -u djnd timeout 3600 /opt/vespene_build/57/vespene_launch.sh as the user djnd it works even in a fresh session. If I try the same as vespene however, it says sudo: a password is required Adding vespene to the sudo group didn't help.

In the end, I made it work, but I'm not sure I know how to automate this in a script. There might be a different way to handle this.

Once vespene was added to the sudo group I had to allow vespene to run sudo without a password. While running the command as djnd it works because the passwords are the same and I guess when it's typed in once it's cached so it doesn't trigger the second time (this is just a wild guess tbh).

If I wanted to run the same command with the vespene user I had to make it able to run the sudo command without the password. There's a setting for that. On a fresh install of Debian 9, this is what you have to do:

  1. read /etc/sudoers.d/README
  2. nano /etc/sudoers.d/vespene (this file will be included in the "main" sudoers file)
  3. add vespene ALL=(ALL) NOPASSWD:ALL to the file, close, and save
  4. as root run visudo this will open a text editor to edit the main file, never edit directly, always use visudo
  5. if you read the README you know what to do, but basically delete the space between # and includedir /etc/sudoers.d on the last line
  6. visudo will parse the file and offer you to edit the README and vespene files, feel free to say yes to double check but you don't have to do anything
  7. after that as root run chmod 0440 /etc/sudoers.d/vespene
  8. visudo -c for one last check and if nothing's wrong you should be good to go

After this I had my first successful build script run. It echoed a message. Now it's time for me to turn it into something more complex. Fingers crossed!


#23

I'm having some reading comprehension trouble after just dealing with software collections in CentOS 7 so bear with me. A few things though:

(A) Password cache should not have any idea that two passwords for two different users are the same.

(B) Vespene shouldn't require usage of nopasswd for sudo isolation. However if you have the OS sudo set to require a password, you must supply one

(C) Configuring sudoers is probably out of scope for our install guide.

(D) I do think I need to take the lesson that I taught myself in ansible and that if there are obvious errors that can occur in specific instances the application, even when unsure of whether that was the situation, does recommend that certain things be tried. I'll be adding a bunch of that soon.

(E) The error you show is an error during a chmod, which implies the sudo user probably doesn't own the build root. I'll see about how that can be made more clear in documentation at least.

Let me know if I'm missing major points/themes, but if I am reading correctly some of the above is you talking yourself through setting up sudo without a password, when I DO think sudo with a password does work fine.

You should not be sudo'ing to Vespene, because that's the user running the worker process - you should be using a different user account for security reasons.

We can also make this be a failure or warning or at least a tooltip in the setup dialog.


#24

Most of what you say is correct. A lst of my writing above is me talking myself through the process of removing the need to run sudo as vespene with a password. However the sudo user defined in the UI (djnd) still requires the password.

Two things, although it's late here, it could just be me being confusing.

(A) The password cache thing could explain why it works if I'm logged in as the worker's "sudo user" (in my case djnd) but doesn't work if you run the command logged in as vespene. It never worked when you have two different users.

(E) I think the log actually shows the error for the sudo_command defined here https://github.com/vespene-io/vespene/blob/dd993c0730b1c9cfa606cf33d7d9561ee9d5ac45/vespene/plugins/isolation/sudo.py#L90 Running this command as vespene (which is what the software does) while requiring vespene to use a password breaks things. Running this as the sudo user defined in the UI (in my case djnd) works. I am not sure why, but allowing vespene to use sudo without a password even though you use sudo as a different user managed to fix things.

I'll ask some more sysadmin-inclined people around me to help me figure out exactly what happened, I'm really not sure if I'm making complete sense ... But it works, so there's that. :P


#25

Ok cool,let me know what you find out. I'll work on adding some sanity checks and so on too.


#26

Here's the ticket on those - https://github.com/vespene-io/vespene/issues/68