Dannielle, how old is Duke?
In my opinion I would be very careful with what Chew Toys you put in the crate. (Make sure it is something he Can NOT break/ choke on if unattended, which is why I am not a huge fan.) There are those rubber toys where you put treats into, if you are gone out a long time you can free meat/ dog food in there and it will keep the puppy busy for some time.
A couple of rules you should implement, make Duke Sit and wait until he is (Calm and Submissive) and then "lure" him into the crate with a treat. Same process for when you bring him out of the crate, and then Reward him when he is out.
Always make sure Duke has gone out/ to the washroom before crating him. Infinitely crating a dog, and not allowing them to expend their energy and release stress is a recipe for disaster.
Teach him the Crate is a GOOD, Safe Place! Let him go in and out of the crate (door open) until he seems more comfortable, then progress to shutting the door. Remember In, Out, In, Out.... Repetition is KEY! Make the time spent in the crate VARY and Random.
NEVER let a Crazy, Hyped-Up, Agitated, Barking, or Crying Dog Out of the Crate. They will associate this with; I Cry or Act Up, equals I get Out.
If you simply can NOT BARE the crying or barking, go stand outside for the 5-10 minutes he is in the crate, go to the store for 20 minutes, put his crate in your garage (if it is secure and stable in weather/temperature) or the basement. The point is, don't give into his Crying! (I know I know, it is easier said than done when a super cute puppy looks* like he is in distress, but keep with it!)
Airline Crates are better than Wire Crates, this is something I have come realize, but it may be more of a personal preference. Dogs are Den Creatures, and these Crates are more 'enclosed' and keeps their hair in better.
Do not try to Praise/ Cuddle/Console the dog when he is acting out, this only ENFORCES the behaviour he is doing at the time of the Praise.
Go on Youtube and watch some videos on Crate Training and Crate Manners, they may help give a better sense of direction if you are more of a visual learner.