The details of the cause and effect are quite intricate:
With one event (summer), another event happened (Peter arrived) which had the following impact (the narrator found it rather trying).
But over time something else happened (the island worked its way under his skin), and caused a change (he became quite human).