Ole Lilloe-Olsen was born into a rich landlord family in Østre Grefsen just outside the old city border of Oslo. He was educated as an engineer, but when his elder brother died from tuberculosis in 1905, he went on to study agriculture for two years. In 1910 he married and took over Vestre Grefsen as a farmer. He was an excellent skier, winning the B-class in Nordic Combined at Holmenkollen in 1915. In 1913 he started his career as a competitive shooter with immediate success, winning a bronze medal in the Norwegain Championships in clay pigeon shooting. In 1918 he won the biggest rifle competition in Norway, Landsskytterstevnet, and was crowned as that year's Shooting King. At the 1920 Olympic Games he competed in running target, winning all three Olympic gold medals. Four years later he added two more Olympic gold medals and a silver to his collection, becoming the most successful Norwegian summer Olympian ever. He continued his shooting career with great success. In 1932, aged 49, he set a world record in clay pigeon with 99 hits out of 100. As late as 1937 he won two gold medals in running target at the Shooting World Championships in Helsinki.