For more information regarding Bristol-Myers Squibb Clinical Trial participation, please visit www.BMSStudyConnect.com Inclusion Criteria: - Histologic or cytologic confirmation of a solid tumor that is advanced (metastatic, recurrent and/or unresectable) - Eastern Cooperative Oncology Group Performance Status of 0 or 1 - Participants must have received, and then progressed, relapsed, or been intolerant to at least 2 standard treatment regimens with proven survival benefit in the advanced or metastatic setting according to tumor type, if such a therapy exists - Advanced stage cutaneous melanoma who have received standard therapies with proven survival benefit including prior immunotherapy with an anti-programmed cell death 1 (anti-PD-1) or anti-programmed death ligand 1 (anti-PD-L1) (For Part 2A) - Non-small cell lung cancer (NSCLC) (adenocarcinoma or squamous cell carcinoma) who have received standard therapies with proven survival benefit including prior immunotherapy with an anti-PD-1 or anti-PD-L1 (For Parts 2B & 2C) - Microsatellite Stable Colorectal Cancer (MSS CRC) who have received standard therapies with proven survival benefit (Part 2D) Exclusion Criteria: - Participants with primary CNS malignancies, or tumors with CNS metastases as the only site of disease, will be excluded - Cytotoxic agents, unless at least 4 weeks have elapsed from last dose of prior anti-cancer therapy and initiation of study therapy - Prior anti-cancer treatments such as chemotherapy, radiotherapy, hormonal, or immunotherapy (including anti-PD-1/PD-L1) are permitted Other protocol-defined inclusion/exclusion criteria apply