The first mentions of the village date back to 1190 or 1234. The tower of the medieval castle which remains testifies to this past like the rue de la Tasco. The village subsequently expanded inside the castle courtyard and then into the suburbs.

Listed and classified site Most Beautiful Villages of France, the hilltop village of Gassin was long called "La Vigie du Golfe". By its position, it was possible from this height to monitor the Gulf of Saint-Tropez - then Gulf of Grimaud - and the Bay of Cavalaire to the Islands of Hyères.
Historical testimonies abound from the medieval past: the Saracen door, opened in the 13th-14th century, the remains of a hinge, or the doors of the long street, the oldest of which dates from 1422.
Rue de la Tasco is the oldest street in the village. It led to the fort of Gassin, of which there remains a one-storey tower, with the bumpy stones characteristic of medieval military architecture in Provence.

Gassin appears - the village is then called Garcin - in several cartularies of the Middle Ages: that of La Verne (1190) and that of the Saint-Victor abbey in Marseille (1234-1235). Previously, there was a fortified villa in Bourrian and a habitat in Ville Vieille.
Despite the presence of a Templar passage, there is nothing to indicate that there was such an establishment. Ancient authors have imagined that the square bell tower of the village church, on the top of the hill overlooking the Gulf of Saint-Tropez, was a Templar lookout.

The habitat in Gassin remains long perched because insecurity reigns for several centuries. The raids of the Saracens continue late: the pirates leave behind the dead and reduce those who are captured in slavery. The kidnapping becomes a source of income for the pirates who sometimes allow the redemption of the captives: several families from Gassin are thus reunited, sometimes after several years of captivity in North Africa. This is the case of one of the two Magnan brothers, kidnapped and detained in Bône.

At the end of the Middle Ages, Saint-Tropez was detached from the seigneury of Gassin.

The 1516 cadastre shows a fortified village built around a single street (today's rue de la Tasco), including the castle and a church, surrounded by a suburb.

During the following centuries, the village gradually extended towards the west, in the Longue, Tubassière, Rollet de Garcin, Moulin à Huile, and Androuno streets.

The village continues to grow with, in particular, appearing on the cadastre of 1728, the hospital to the north and the forge to the south. The fortified enclosure then has two entrances, one to the north, through the New Portal, the other to the west with the Grand Portal.

In the XIXth century, the castral borough almost reached its current composition.