Heat olive oil in a large Dutch oven set over medium-high heat. Add the onion and saute for 5 minutes, until softened.
Stir in the peeled, chopped potatoes and season with salt and pepper to taste. Add broth to just cover the potatoes and bring them to a boil. Reduce heat to medium, cover, and simmer for 20 minutes, until the potatoes are fork tender. Remove the pot from the heat and stir in the milk (or cream).
Using an immersion blender, blend the soup until smooth. Alternatively, you can place contents in a blender until smooth. Taste and adjust seasonings as desired.
Stir in any additional optional ingredients until well combined, and serve with desired toppings.