In a large saucepan, combine the milk, 1/2 cup of the sugar, and vanilla bean and seeds, and bring to a gentle boil. Remove from the heat.

In a medium bowl, beat the yolks until pale yellow and thick, 2 to 3 minutes. Slowly pour in 1 cup of the hot milk, whisking constantly. Pour the egg yolk mixture into the saucepan with the hot milk, whisking. Cook over medium heat, stirring constantly with a heavy wooden spoon, until the mixture thickens enough to coat the back of a spoon, about 5 minutes. Strain through a fine mesh strainer into a clean container. Cover with plastic wrap, pressing down against the surface to keep a skin from forming. Refrigerate until well chilled, at least 4 hours or overnight.

In a large bowl, beat the egg whites with an electric mixer until soft peaks start to form. Slowly add 1/4 cup of the sugar and beat until stiff peaks form.

In a large bowl with clean beaters, beat the cream until thick and frothy. Add the remaining 1/4 cup sugar and beat to soft peaks.

In a large punch bowl, combine the chilled egg yolk mixture and the bourbon and brandy. Fold in the sweetened whipped cream, then the egg whites. Refrigerate until chilled, 30 minutes to 1 hour.

Sprinkle nutmeg over the top and garnish with chocolate shavings. Serve chilled.