Vulkan queue waiting on semaphore that can’t be signaled

It seems I have had invalid code for a while but the validation layers were silent. After updating my sdk to the latest version I started getting this error:

Message ID name: VUID-vkQueuePresentKHR-pWaitSemaphores-03268
Message: [ VUID-vkQueuePresentKHR-pWaitSemaphores-03268 ] Object: 0x55b4b87478f0 (Name = Selected logical device : Type = 3) | VkQueue 0x55b4b8224020[Main queue] is waiting on VkSemaphore 0x110000000011[Render Finished Semaphore: 0] that has no way to be signaled. The Vulkan spec states: All elements of the pWaitSemaphores member of pPresentInfo must reference a semaphore signal operation that has been submitted for execution and any semaphore signal operations on which it depends (if any) must have also been submitted for execution. (https://www.khronos.org/registry/vulkan/specs/1.1-extensions/html/vkspec.html#VUID-vkQueuePresentKHR-pWaitSemaphores-03268)
Severity: VK_DEBUG_UTILS_MESSAGE_SEVERITY_ERROR_BIT_EXT

This happens inside of my main draw loop and it's the only validation layer error in my code. If I never call the code that is responsible for surface presentation. I get no errors.

I am sure the place where I am doing things wrong is here:

void DisplayTarget::StartPass(uint target_num, bool should_clear,
    VulkanImage* external_depth)
{
    auto device = h_interface->GetDevice();

    auto result = device.acquireNextImageKHR(
        *swap_chain,
        std::numeric_limits<uint64_t>::max(),
        *img_available_sems[current_frame],
        nullptr,
        &active_image_index);

    if(result != vk::Result::eSuccess)
        Log::RecordLog("Failed to acquire image");
}

vk::Result DisplayTarget::EndPass()
{
    auto device         = h_interface->GetDevice();
    auto cmd_buff       = h_interface->GetCmdBuffer();
    auto graphics_queue = h_interface->GetQueue();

    device.waitForFences(
        1,
        &*in_flight_fences[current_frame],
        VK_TRUE,
        std::numeric_limits<uint64_t>::max());

    vk::Semaphore wait_semaphores[]      = {*img_available_sems[current_frame]};
    vk::PipelineStageFlags wait_stages[] = {
        vk::PipelineStageFlagBits::eColorAttachmentOutput};
    vk::Semaphore signal_semaphores[] = {*render_finished_sems[current_frame]};
    vk::SubmitInfo submit_info(
        1, wait_semaphores, wait_stages, 1, &cmd_buff, 1, signal_semaphores);

    device.resetFences(1, &*in_flight_fences[current_frame]);

    auto result =
        graphics_queue.submit(1, &submit_info, *in_flight_fences[current_frame]);
    if(result != vk::Result::eSuccess)
        Log::RecordLog("Failed to submit draw command buffer!");

    graphics_queue.waitIdle();
    device.waitIdle();

    vk::SwapchainKHR swap_chains[] = {*swap_chain};
    vk::PresentInfoKHR present_info = {};
    present_info.waitSemaphoreCount = 1;
    present_info.pWaitSemaphores = signal_semaphores;
    present_info.swapchainCount = 1;
    present_info.pSwapchains = swap_chains;
    present_info.pImageIndices = &active_image_index;

    result = graphics_queue.presentKHR(&present_info);

    current_frame = (current_frame + 1) % MAX_FRAMES_IN_FLIGHT;

    return result;
}

Currently they are called consecutively:

display.StartPass();        
display.EndPass();

To get things working I have tried both commenting parts of these 2 functions out or changing the order in which things are called, but either the error persists or I get different validation errors.

I also tried signaling the semaphore directly:

vk::SemaphoreSignalInfo semaphore_info = {};
semaphore_info.semaphore = *render_finished_sems[current_frame];
semaphore_info.value = 0;
device.signalSemaphore(semaphore_info);

But all I managed is to cause a segmentation fault