In a medium bowl, cream together the butter, sugar, and vanilla for 2-3 minutes until fluffy. Scrape down the sides of the bowl, then add the flour and salt and mix until incorporated.
Cover the bowl with plastic wrap and chill for at least 1 hour.
When ready to bake, preheat the oven to 350 degrees F. Line 2 baking sheets with parchment paper and set them aside.
Roll the dough into 1 ½-inch balls, and place 1 inch apart on the prepared baking sheets. Use your thumb or the back of a round measuring spoon to press a small hole in the center of each ball.
Bake them for 10 minutes, until slightly firm, then remove the baking sheets from the oven. Use the spoon to press into the indentations in each cookie again, then return to the oven and bake them for an additional 6-8 minutes, or until lightly browned. Let cool 1 minute on the baking sheets, then transfer to a wire rack to cool completely.
In a small bowl, whisk the powdered sugar, milk, and vanilla together in a small bowl until smooth. Divide the icing in half and add a few drops of food coloring to each, whisking to incorporate. Add more milk if needed, but not so much that it becomes runny. Pour icing into plastic zipper bags, snip off a corner, and fill the center of each cookie. Allow to dry and set overnight.