Preheat the oven to 350 F, and position a rack in the center of the oven. Line a 9-inch square baking dish with parchment paper, or spray it with non-stick baking spray.
In a saucepan, melt the butter over medium heat. Continue to cook the butter until nutty, golden brown milk solids form at the bottom of the pan. This process might take about 10 minutes. First the butter will hiss and splatter as the water evaporates, then as it quiets down, a layer of foam will form on top of the butter. This means the butter is done. Immediately remove from the heat, and scrape the butter, including the brown milk solids, into a large bowl. Let cool for 10 minutes.
In a bowl, whisk together the cooled brown butter with the granulated sugar, eggs, vanilla extract, almond extract, milk and sour cream, until combined.
In a separate bowl, sift together the flour, salt, baking powder and cardamom. Add the almond flour, then whisk until everything is evenly distributed.
Add the dry ingredients to the wet, whisking just until moistened.
Pour the batter into the prepared pan, and bake for about 32-35 minutes, until the top springs back when gently touched, or a toothpick or cake tester comes out clean. Set the pan on a wire rack, covered with a clean kitchen towel, and cool completely before frosting.
This cake will have a golden brown crust around the edges, very much like a pound cake, and it's absolutely delicious. If you prefer, you can trim off the dark edge before serving.