I’m adding new fields programmatically. How do I set the default visibility?

$field_id = xprofile_insert_field([
    'field_group_id'    => $group->id,
    'type'              => 'textbox',
    'name'              => 'Admins Only Field',
    'is_required'       => false,
    'can_delete'        => true,
    'field_order'       => 3,
    'description'       => "Enter something in this field that only you can see."
]);

After getting the $field_id, I can load all the users and set their individual visibility to adminsonly using xprofile_set_field_visibility_level — but that doesn’t change it on the Edit Field info.