Inclusion Criteria: - Histological confirmed previously treated colorectal cancer with adenocarcinoma histology with metastatic or recurrent unresectable disease at study entry - Must have historically or locally confirmed tumor microsatellite stability (stable) (MSS) / proficient mismatch repair (pMMR) status - Participants must have: 1. progressed during or within approximately 3 months following the last administration of approved standard therapies (at least 1, but not more than 4 prior lines of therapies), which must include a fluoropyrimidine, oxaliplatin, irinotecan, an anti-VEGF therapy, and anti-EGFR therapy (if KRAS wild-type), if approved in the respective country, or; 2. been intolerant to prior systemic chemotherapy regimens if there is documented evidence of clinically significant intolerance despite adequate supportive measures - Must have sufficient tumor tissue & evaluable PD-L1 expression to meet the study requirements - Must have measurable disease per RECIST v1.1. Participants with lesions in a previously irradiated field as the sole site of measurable disease will be permitted to enroll provided the lesion(s) have demonstrated clear progression and can be measured accurately Exclusion Criteria: - Prior treatment with either an immunotherapy or with regorafenib or with TAS-102 - Untreated central nervous system (CNS) metastases, participants are eligible if CNS metastases have been treated and participants have neurologically returned to baseline (except for residual signs or symptoms related to the CNS treatment) - History of refractory hypertension not controlled with anti-hypertensive therapy, myocarditis (regardless of etiology), uncontrolled arrhythmias, acute coronary syndrome within 6 months prior to dosing, Class II congestive heart failure (as per the New York Heart Association Functional Classification), interstitial lung disease/pneumonitis or an active, known or suspected autoimmune disease Other protocol-defined inclusion/exclusion criteria apply