In a large mixing bowl, beat the cream cheese with an electric mixer until completely smooth and creamy, with no lumps remaining.
Add the pumpkin puree, brown sugar, vanilla extract, and pumpkin pie spice. Continue mixing until the ingredients are fully incorporated and the mixture is light and fluffy.
To assemble the trifle, place half of the cubed angel food cake evenly across the bottom of a trifle bowl.
Spoon half of the pumpkin cheesecake mixture over the cake, spreading it into an even layer.
Spread half of the Cool Whip on top of the pumpkin layer.
Repeat the layers with the remaining cake cubes, pumpkin cheesecake mixture, and Cool Whip.
Lightly dust the top with pumpkin pie spice for garnish.
Cover and refrigerate for 1-2 hours, or until well chilled and ready to serve.