The general opinion in the comments both here and on my own blog was (like I thought) that I could have started with size 12 in the shoulder/neck part. Based on a comment on my blog, I first tried to change the muslin I already had instead of sewing a complete new muslin. Below the new pictures, again only the right side of the jacket is changed.
I'm wearing a thin lambswool top underneath, and think I won't wear anything with more thickness under this jacket. It feels a bit snug, but like Lindsay T said, this is meant to be a snug fitted jacket. Am I ready to cut in the real fabric?
On the right is the new adapted version. For comparison I have put the pictures side by side.