Beat egg yolks, add sugar and beat again. Mix Rusk crumbs, nuts, vanilla, and baking powder. Fold in egg whites. Put in a greased 9 x 13-inch pan and bake at 350 degrees for 30 minutes; cool.
Melt Milk Duds, mix in milk, powdered sugar, and butter. Whip cream and pour over baked torte. Cover with Milk Dud sauce and refrigerate overnight.