Social norms in group dynamics refer to the unwritten rules and expectations that dictate behavior within the group.