Additionally, a traditional Allen key has a built in torque setting in use. When nearly tight, use just your finger right at the end of the lever and it'll start to bend (springy) just as it reaches the tight enough point, the reason they are all different lengths to facilitate this.
For the OP all bolts made of steel into aluminium threads will ultimately strip the aluminium threads out if too tight, and so you're right to be careful in assembly of this.