Skip to content

KKS - Posibility of making a 180 screenshot? #213

Open
@jasonbarthes

Description

@jasonbarthes

Hello! I've been using 360 screenshot to generate vids. However the file size and VRAM requirement makes the generation risky, which means that my PC always crashed during the process.
Recently I saw someone shared a modified dll for KK's screencap, which is able to capture 180 screenshots:(https://www.iwara.tv/post/20hnPDq0emf58K)
I tried that in KK and the alising was way better than 360, under the same resolution setting. Because my data are compatible with KKS only, I want to make a kks version plugin. Do you have any idea about how to turn the KKS plugin into a 180 version? I can code and contribute if you don't mind.

Activity

ManlyMarco

ManlyMarco commented on Jan 19, 2025

@ManlyMarco
Collaborator

If someone makes a PR that adds this feature then by all means I will merge it.

I made an attempt at merging the changes made in that modified dll here https://github.com/IllusionMods/BepisPlugins/tree/3d-180 but got stuck on something, I don't remember what exactly was the issue. At the moment I don't plan on working on this any further.

mellona0821

mellona0821 commented on Jan 25, 2025

@mellona0821

@jasonbarthes
I’m not sure if it’s okay to write this here, but since I received a request, I’ll share it here
I was considering preparing a PR myself, but it seems there’s more to take care of than I expected.
Instead, I’ll just share the parts I modified here.
It’s been quite a while, so I don’t remember everything exactly, but I’ll do my best to write down as much as I can.

First, to change the 360-degree view to 180 degrees and apply tilt, I modified the shader at the following link:
https://github.com/mellona0821/BepisPlugins/blob/master/src/Core_Screencap/shader_source/EquirectangularConverter.shader

Shader "Hidden/I360CubemapToEquirectangular"
{
    Properties
    {
        _MainTex ("Cubemap (RGB)", CUBE) = "" {}
        _CameraRotationMatrix("Camera Rotation Matrix", Matrix) = (1,0,0,0,0,1,0,0,0,0,1,0,0,0,0,1)
    }

    Subshader
    {
        Pass
        {
            ZTest Always Cull Off ZWrite Off

            CGPROGRAM
            #pragma vertex vert
            #pragma fragment frag
            #pragma fragmentoption ARB_precision_hint_fastest
            #include "UnityCG.cginc"

            #define PI 3.141592653589793
            #define TWOPI 6.283185307179587

            struct v2f
            {
                float4 pos : POSITION;
                float2 uv : TEXCOORD0;
            };

            samplerCUBE _MainTex;
            float4x4 _CameraRotationMatrix;

            v2f vert(appdata_img v)
            {
                v2f o;
                o.pos = UnityObjectToClipPos(v.vertex);
                o.uv = v.texcoord.xy * float2(TWOPI, PI);
                return o;
            }

            fixed4 frag(v2f i) : COLOR 
            {
                float theta = i.uv.y;
                float phi = i.uv.x;

                float3 unit = mul(_CameraRotationMatrix, float4(
                    sin(phi) * sin(theta) * -1,
                    cos(theta) * -1,
                    cos(phi) * sin(theta) * -1, 
                    0
                )).xyz;

                return texCUBE(_MainTex, unit);
            }
            ENDCG
        }
    }
    Fallback Off
}

To match the modified shader, I also updated the code at the following link accordingly:
https://github.com/mellona0821/BepisPlugins/blob/master/src/Core_Screencap/Renderers/I360Render.cs

public static Texture2D CaptureTex(int width = 1024, Camera renderCam = null, bool faceCameraDirection = true)
{
	if (renderCam == null)
	{
		renderCam = Camera.main;
		if (renderCam == null)
		{
			throw new Exception("No camera detected");
		}
	}
	RenderTexture targetTexture = renderCam.targetTexture;
	int num = Mathf.Min(Mathf.NextPowerOfTwo(width), 20000);
	RenderTexture renderTexture = null;
	RenderTexture renderTexture2 = null;
	Texture2D result;
	try
	{
		renderTexture = RenderTexture.GetTemporary(num, num, 0, 0);
		renderTexture.dimension = 4;
		renderTexture2 = RenderTexture.GetTemporary(num, num, 0, 0);
		renderTexture2.dimension = 2;
		if (!renderCam.RenderToCubemap(renderTexture))
		{
			throw new Exception("Rendering to cubemap is not supported on device/platform");
		}
		I360Render.equirectangularConverter.SetFloat(I360Render.paddingX, faceCameraDirection ? (renderCam.transform.eulerAngles.y / 360f) : 0f);
		Matrix4x4 localToWorldMatrix = renderCam.transform.localToWorldMatrix;
		I360Render.equirectangularConverter.SetMatrix(I360Render.cameraRotationMatrixID, localToWorldMatrix);
		Graphics.Blit(renderTexture, renderTexture2, I360Render.equirectangularConverter);
		result = I360Render.RtToT2D(renderTexture2);
	}
	finally
	{
		renderCam.targetTexture = targetTexture;
		if (renderTexture != null)
		{
			RenderTexture.ReleaseTemporary(renderTexture);
		}
		if (renderTexture2 != null)
		{
			RenderTexture.ReleaseTemporary(renderTexture2);
		}
	}
	return result;
}

Even though it worked fine with just these changes, I analyzed the code to improve performance and found that the part where the two images (for the left and right eye) are combined was quite inefficient.
So, I combined the capturing and image-stitching code into a single method.

public static Texture2D CaptureStereoTex(int width = 1024, Camera renderCam = null, bool faceCameraDirection = true, float eyeSeparation = 0.6f, bool flipEyes = false)
{
	if (renderCam == null)
	{
		renderCam = Camera.main;
		if (renderCam == null)
		{
			throw new Exception("No camera detected");
		}
	}
	RenderTexture targetTexture = renderCam.targetTexture;
	int num = Mathf.Min(Mathf.NextPowerOfTwo(width), 20000);
	RenderTexture renderTexture = null;
	RenderTexture renderTexture2 = null;
	RenderTexture renderTexture3 = null;
	RenderTexture renderTexture4 = null;
	Texture2D result;
	try
	{
		renderTexture = RenderTexture.GetTemporary(num, num, 0, 0);
		renderTexture.dimension = 4;
		renderTexture2 = RenderTexture.GetTemporary(num, num, 0, 0);
		renderTexture2.dimension = 2;
		renderTexture3 = RenderTexture.GetTemporary(num, num, 0, 0);
		renderTexture4 = RenderTexture.GetTemporary(num, num, 0, 0);
		I360Render.equirectangularConverter.SetFloat(I360Render.paddingX, faceCameraDirection ? (renderCam.transform.eulerAngles.y / 360f) : 0f);
		I360Render.equirectangularConverter.SetMatrix(I360Render.cameraRotationMatrixID, renderCam.transform.localToWorldMatrix);
		renderCam.transform.position += renderCam.transform.right * (eyeSeparation / 2f);
		if (!renderCam.RenderToCubemap(renderTexture))
		{
			throw new Exception("Rendering to cubemap is not supported on device/platform");
		}
		Graphics.Blit(renderTexture, renderTexture4, I360Render.equirectangularConverter);
		renderCam.transform.position -= renderCam.transform.right * eyeSeparation;
		if (!renderCam.RenderToCubemap(renderTexture))
		{
			throw new Exception("Rendering to cubemap is not supported on device/platform");
		}
		Graphics.Blit(renderTexture, renderTexture3, I360Render.equirectangularConverter);
		renderCam.transform.position += renderCam.transform.right * (eyeSeparation / 2f);
		Texture2D texture2D = new Texture2D(renderTexture2.width * 2, renderTexture2.height, 5, false);
		RenderTexture.active = (flipEyes ? renderTexture4 : renderTexture3);
		texture2D.ReadPixels(new Rect(0f, 0f, (float)renderTexture2.width, (float)renderTexture2.height), 0, 0);
		RenderTexture.active = (flipEyes ? renderTexture3 : renderTexture4);
		texture2D.ReadPixels(new Rect(0f, 0f, (float)renderTexture2.width, (float)renderTexture2.height), renderTexture2.width, 0);
		texture2D.Apply();
		result = texture2D;
	}
	finally
	{
		renderCam.targetTexture = targetTexture;
		if (renderTexture != null)
		{
			RenderTexture.ReleaseTemporary(renderTexture);
		}
		if (renderTexture2 != null)
		{
			RenderTexture.ReleaseTemporary(renderTexture2);
		}
		if (renderTexture3 != null)
		{
			RenderTexture.ReleaseTemporary(renderTexture3);
		}
		if (renderTexture4 != null)
		{
			RenderTexture.ReleaseTemporary(renderTexture4);
		}
	}
	return result;
}

I originally modified this for personal use, so the code isn’t exactly high quality.
Also, since I rewrote this based on memory, it might not be entirely accurate.
Still, I hope it can be helpful in some way!

jasonbarthes

jasonbarthes commented on Feb 17, 2025

@jasonbarthes
Author

Thank you for the response. But I'm stuck where I try to compile the plugin

jasonbarthes

jasonbarthes commented on Feb 17, 2025

@jasonbarthes
Author

public static Texture2D CaptureStereoTex(int width = 1024, Camera renderCam = null, bool faceCameraDirection = true, float eyeSeparation = 0.6f, bool flipEyes = false)
{
if (renderCam == null)
{
renderCam = Camera.main;
if (renderCam == null)
{
throw new Exception("No camera detected");
}
}
RenderTexture targetTexture = renderCam.targetTexture;
int num = Mathf.Min(Mathf.NextPowerOfTwo(width), 20000);
RenderTexture renderTexture = null;
RenderTexture renderTexture2 = null;
RenderTexture renderTexture3 = null;
RenderTexture renderTexture4 = null;
Texture2D result;
try
{
renderTexture = RenderTexture.GetTemporary(num, num, 0, 0);
renderTexture.dimension = 4;
renderTexture2 = RenderTexture.GetTemporary(num, num, 0, 0);
renderTexture2.dimension = 2;
renderTexture3 = RenderTexture.GetTemporary(num, num, 0, 0);
renderTexture4 = RenderTexture.GetTemporary(num, num, 0, 0);
I360Render.equirectangularConverter.SetFloat(I360Render.paddingX, faceCameraDirection ? (renderCam.transform.eulerAngles.y / 360f) : 0f);
I360Render.equirectangularConverter.SetMatrix(I360Render.cameraRotationMatrixID, renderCam.transform.localToWorldMatrix);
renderCam.transform.position += renderCam.transform.right * (eyeSeparation / 2f);
if (!renderCam.RenderToCubemap(renderTexture))
{
throw new Exception("Rendering to cubemap is not supported on device/platform");
}
Graphics.Blit(renderTexture, renderTexture4, I360Render.equirectangularConverter);
renderCam.transform.position -= renderCam.transform.right * eyeSeparation;
if (!renderCam.RenderToCubemap(renderTexture))
{
throw new Exception("Rendering to cubemap is not supported on device/platform");
}
Graphics.Blit(renderTexture, renderTexture3, I360Render.equirectangularConverter);
renderCam.transform.position += renderCam.transform.right * (eyeSeparation / 2f);
Texture2D texture2D = new Texture2D(renderTexture2.width * 2, renderTexture2.height, 5, false);
RenderTexture.active = (flipEyes ? renderTexture4 : renderTexture3);
texture2D.ReadPixels(new Rect(0f, 0f, (float)renderTexture2.width, (float)renderTexture2.height), 0, 0);
RenderTexture.active = (flipEyes ? renderTexture3 : renderTexture4);
texture2D.ReadPixels(new Rect(0f, 0f, (float)renderTexture2.width, (float)renderTexture2.height), renderTexture2.width, 0);
texture2D.Apply();
result = texture2D;
}
finally
{
renderCam.targetTexture = targetTexture;
if (renderTexture != null)
{
RenderTexture.ReleaseTemporary(renderTexture);
}
if (renderTexture2 != null)
{
RenderTexture.ReleaseTemporary(renderTexture2);
}
if (renderTexture3 != null)
{
RenderTexture.ReleaseTemporary(renderTexture3);
}
if (renderTexture4 != null)
{
RenderTexture.ReleaseTemporary(renderTexture4);
}
}
return result;
}

I tried these steps but the result was the code wants to access properties that are not accessible for Unity 5.6.x.(eg.RenderTexture.dimension). I could not compile it until I replaced these lines. However, the dll I generated was still giving me 360 results

mellona0821

mellona0821 commented on Mar 14, 2025

@mellona0821

Are you still working on the issue?
If so, could you give this one a try?

public static Texture2D CaptureStereoTex(int width = 1024, Camera renderCam = null, bool faceCameraDirection = true, float eyeSeparation = 0.6f, bool flipEyes = false)
{
  if (renderCam == null)
  {
    renderCam = Camera.main;
    if (renderCam == null)
    {
      throw new Exception("No camera detected");
    }
  }
  RenderTexture targetTexture = renderCam.targetTexture;
  int num = Mathf.NextPowerOfTwo(width);
  RenderTexture renderTexture = null;
  RenderTexture renderTexture2 = null;
  RenderTexture renderTexture3 = null;
  Texture2D result;
  try
  {
    renderTexture = RenderTexture.GetTemporary(num, num, 0, RenderTextureFormat.ARGB32);
    renderTexture2 = RenderTexture.GetTemporary(num, num, 0, RenderTextureFormat.ARGB32);
    renderTexture3 = RenderTexture.GetTemporary(num, num, 0, RenderTextureFormat.ARGB32);
    renderTexture.dimension = TextureDimension.Cube;
    I360Render.equirectangularConverter.SetFloat(I360Render.paddingX, faceCameraDirection ? (renderCam.transform.eulerAngles.y / 360f) : 0f);
    I360Render.equirectangularConverter.SetMatrix(I360Render.cameraRotationMatrixID, renderCam.transform.localToWorldMatrix);
    renderCam.transform.position += renderCam.transform.right * (eyeSeparation / 2f);
    if (!renderCam.RenderToCubemap(renderTexture))
    {
      throw new Exception("Rendering to cubemap is not supported on device/platform");
    }
    Graphics.Blit(renderTexture, renderTexture3, I360Render.equirectangularConverter);
    renderCam.transform.position -= renderCam.transform.right * eyeSeparation;
    if (!renderCam.RenderToCubemap(renderTexture))
    {
      throw new Exception("Rendering to cubemap is not supported on device/platform");
    }
    Graphics.Blit(renderTexture, renderTexture2, I360Render.equirectangularConverter);
    renderCam.transform.position += renderCam.transform.right * (eyeSeparation / 2f);
    Texture2D texture2D = new Texture2D(num * 2, num, TextureFormat.ARGB32, false);
    RenderTexture.active = (flipEyes ? renderTexture3 : renderTexture2);
    texture2D.ReadPixels(new Rect(0f, 0f, (float)num, (float)num), 0, 0);
    RenderTexture.active = (flipEyes ? renderTexture2 : renderTexture3);
    texture2D.ReadPixels(new Rect(0f, 0f, (float)num, (float)num), num, 0);
    texture2D.Apply();
    result = texture2D;
  }
  finally
  {
    renderCam.targetTexture = targetTexture;
    RenderTexture.active = null;
    if (renderTexture != null)
    {
      RenderTexture.ReleaseTemporary(renderTexture);
    }
    if (renderTexture2 != null)
    {
      RenderTexture.ReleaseTemporary(renderTexture2);
    }
    if (renderTexture3 != null)
    {
      RenderTexture.ReleaseTemporary(renderTexture3);
    }
  }
  return result;
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

      Participants

      @ManlyMarco@jasonbarthes@mellona0821

      Issue actions

        KKS - Posibility of making a 180 screenshot? · Issue #213 · IllusionMods/BepisPlugins