The building unfolds across the landscape, with several halls linked to the main entrance and the now iconic central round transparent plastic tube elevator, which served as the inspiration for HOK's circular logo. The exhibition spaces open out towards the fjord like a fan, a stretched hand or an unfurled maple leaf. The architects claimed Frank Lloyd Write, Le Corbusier and Mies van der Rohe as their main sources of inspiration.

Henie and Onstad opted for a team of talented, young Norwegian partners, and they exclusively used Norwegian materials and suppliers throughout the construction process. The Kunstsenter's first director, Ole Henrik Moe, exerted sizable influence on the process, jointly with the two architects and the founders. The building was crafted specifically to house the visual arts, a restaurant, as well as music and performance arts. Two handmade, state-of-the-art mixing boards from Kongsberg Våpenfabrikk helped to establish HOK as one of Europe's most advanced recording studios.

In 1994, the building was extended with a two-story wing along the waterfront, comprising what was then Norway's largest exhibition space (1,300 square meters), as well as several technical spaces and workshops. The wing was designed by the same architects, Eikvar and Engebretsen, and is constructed as a natural extension of the original building. In 2003, another extension was built, which was donated by the gallery owner Haaken A. Christen and designed by architect Stein Halvorsen. The main material used in the construction of the extension was wood, in contrast to the main building's primary use of concrete and aluminum.

Altogether, the current building covers a space of approximately 9,500 square meters, of which approximately 3,500 are for exhibition space. In 2013, HOK saw the start of a major renovation project, with the aim to restore many of the original qualities of the main building from 1968, and make it more energy efficient and sustainable for the future. The restoration work will continue through 2017.