Orthogonal moments (OMs) are used to extract features from color images. OMs with frac- tional orders are better than the OMs with integer orders due to their ability to extract fine features. This paper defined novel quaternion orthogonal shifted Gegenbauer moments (FrQSGMs) of fractional orders for color image analysis and recognition. Since both shifted Gegenbauer polynomials and the input digital images are defined in the domain [0, 1] ×[0, 1], the proposed FrQSGMs did not need any image mapping or image interpolation. The invariance to geometric transformations of the proposed FrQSGMs is derived by express- ing these moments in geometric moment invariants of fractional order. We conduct var- ious experiments to test the accuracy, invariance to RST, sensitivity to noise, recognition of similar color images, and computational times. The proposed descriptors outperformed the existing orthogonal moments with fractional orders.