Here is the information from tvshowsondvd.com:


Finally....the news! - 7:07 PM 2/19/2003
We reported this back in October, but this is official news coming from Warner Bros. "The Ben Stiller Show" will be released on June 3rd for a suggested retail price of $34.98. The DVDs will contain all 13 episodes on 2 discs with Dolby Surround Stereo audio and extras. We'll post more details as they become available.

I found a news report about the extras that will be included on the set:

And among the extras are 7 audio commentary tracks with the entire cast (including Ben Stiller and the writers), unaired sketches (with optional commentary), 10 outtakes, the A Brief History of The Ben Stiller Show featurette, the E! Behind the Scenes special, alternative versions of the pilot episode and the Melrose Heights special commentary Easter egg. SRP is $34.98.

That's good news. Another briliant show cancelled by the geniuses at FOX.