The Queer Lion award, a take on the festival's main Golden Lion prize, will go to the best full-length film featuring a gay theme or character.
Competition director Daniel Casagrande said it had taken four years of negotiations to introduce the award.
He has said he expects "around 10 or 12" films to be candidates for the inaugural prize.
He added: "We aren't looking for the next Brokeback Mountain. We are just looking for films that accurately portray gay characters or themes."
Films entering the main competition are eligible for the award, as are other films being shown at the festival, which runs from 29 August to 8 September.
A small international jury will select the winner, who will be presented with a gold plaque featuring the Venice Lion winged logo with the rainbow colours of the gay pride flag on the wings.
The Berlin International Film Festival has been handing out a similar prize, the Teddy Award, for the past 21 years as an offshoot of its Golden Bear award.