For more information regarding Bristol Myers Squibb Clinical Trial participation, please visit www.BMSStudyConnect.com Inclusion Criteria: - Must have previously treated metastatic colorectal cancer - Must have RAS mutation and microsatellite stability status results as part of medical history - Must agree to provide archival or newly obtained tumor tissue sample prior to the start of treatment in this study - Eastern Cooperative Oncology Group (ECOG) performance status of ≤1 - Ability to swallow pills or capsules - Adequate organ functions - Ability to comply with study visits, treatment, procedures, PK and PD sample collection, and required study follow-up Exclusion Criteria: - BRAF V600 mutant colorectal cancer - Active brain metastases or leptomeningeal metastases - Active, known or suspected autoimmune disease - Histology other than adenocarcinoma - Participants with a condition requiring systemic treatment with either corticosteroids (> 10 mg daily prednisone equivalents) or other immunosuppressive medications within 14 days of study treatment administration - History of interstitial lung disease or pneumonitis - Prior treatment with immune checkpoint inhibitors and MEK inhibitors - History of allergy to study treatments or any of its components of the study arm that participant is enrolling Other protocol defined inclusion/exclusion criteria could apply