The first question concerns the oceans edge card Davy Jones, which states, “limit. Once per turn, roll a D6. On a 1–4 you may give the ship an extra action: on a 5, you may give any ship you control an extra action: on a 6, you may give an enemy ship an action.”
I assume this means if you roll a 6, you can move an enemy 5 masted ship into a reef to do damage?
Yes, that’s one of many options you can do with that ability.
Second question. What I said above actually happened in the game. The five masted ship went into a reef by Davy Jones effect, and lost all its masts eventually. The question is, when a ship loses all its masts inside a reef, does that mean you remove half of the hull to show that it is shipwrecked? Can a shipwreck inside a reef be explored by any player?
The ship only becomes a wreck if it would take more damage than it has masts remaining. Essentially, if the reef would have done enough damage to sink it. If it takes exactly as much damage as it has mast remaining it only becomes a derelict, and can still be repaired (or towed).
If a ship does become a wreck, it can be explored by any player.