A stock F10 M5 is not trapping that high but closer to 121. It also weighs 1000+ pounds more than the GT3 RS.
Now, the M5 is claimed to be stock but is it? That is not the factory exhaust system on the car. Additionally, this Youtube channel is well known for racing modifying BMW M cars against stock cars and claiming the BMW's are stock. The Porsche owner supposedly stated the F10 M5 is modified but the M5 owner supposedly claims he is stock.
To further add to the puzzle, this same GT3 RS demolishes a new Cadillac CTS-V LT4 in another video. It isn't even close. How does the GT3 RS go from stomping a car offering similar performance and weight to the M5 to suddenly struggling? It may have something to do with an M5 being involved in the race rather than a CTS-V.
Yes, the GT3 RS aerodynamics will cause drag at higher speeds. Yes, the M5 is more powerful. Yes, you should question this result and previous results in this videos.