Bring milk, sugar, and pumpkin purée to simmer in a medium saucepan on medium heat, stirring to dissolve sugar.
Carefully pour mixture into blender container. Add coffee, vanilla, and pumpkin pie spice.
With the center part of blender cover removed to let steam escape, blend on medium speed until smooth and frothy. Pour into mugs.
Top with whipped cream and sprinkle with additional pumpkin pie spice, if desired.