Preheat the oven to 350°F and prepare a muffin tin with paper or silicone liners.
Next, combine the flour, cinnamon, baking powder, baking soda, and salt in a large mixing bowl.
Mix the pumpkin puree, brown sugar, oil, vanilla extract, milk, and eggs in a separate bowl until well combined.
Pour the wet ingredients into the dry ingredients and stir until just combined. Be careful not to overmix, add chopped walnuts and mix just until combined.
Spoon the batter into the prepared muffin tin, filling each cup about 2/3 full.
Bake for 20-25 minutes, or until a toothpick inserted into the center of a muffin comes out clean.
Allow the muffins to cool in the tin for a few minutes before transferring them to a wire rack to cool completely.
Once the muffins are completely cool, top with icing or sprinkle with powdered sugar before serving.
To make the icing, add the powdered sugar to a small bowl. Gradually add the milk, starting with 2 tablespoons, and whisk until smooth. If the icing is too thick, add more milk, a little at a time, until you reach your desired consistency.
Once the pumpkin muffins have cooled completely, drizzle the icing over the tops of the muffins. If desired, add more chopped walnuts on top of the icing.
Allow the icing to set before serving the muffins.