Connect jan to remote cortex?

[MalfuncEddie]

Is it possible with Jan to do the following

Run a cortex server on something that has a GPU
Make Jan connect to it via a vpn.

Since cortex is openapi compatible I should be ale to connect it ?
Any one got a guide for splitting Jan/ cortex over 2 machines?

[MalfuncEddie]

When clicking refresh it does not show the models eventough the endpoit does work

{
	"data" : 
	[
		{
			"ai_template" : "<|im_end|>\n<|im_start|>assistant\n",
			"created" : 140538820899632,
			"ctx_len" : 4096,
			"dynatemp_exponent" : 1.0,
			"dynatemp_range" : 0.0,
			"engine" : "llama-cpp",
			"files" : 
			[
				"models/cortex.so/qwen3/4b/model.gguf"
			],
			"frequency_penalty" : 0.0,
			"gpu_arch" : "",
			"id" : "qwen3:4b",
			"ignore_eos" : false,
			"max_tokens" : 4096,
			"min_keep" : 0,
			"min_p" : 0.05,
			"mirostat" : false,
			"mirostat_eta" : 0.1,
			"mirostat_tau" : 5.0,
			"model" : "qwen3:4b",
			"n_parallel" : 1,
			"n_probs" : 0,
			"name" : "qwen3:4b",
			"ngl" : 37,
			"object" : "",
			"os" : "",
			"owned_by" : "",
			"penalize_nl" : false,
			"precision" : "",
			"presence_penalty" : 0.0,
			"prompt_template" : "<|im_start|>system\n{system_message}<|im_end|>\n<|im_start|>user\n{prompt}<|im_end|>\n<|im_start|>assistant\n",
			"quantization_method" : "",
			"repeat_last_n" : 64,
			"repeat_penalty" : 1.0,
			"seed" : -1,
			"size" : 2497281513,
			"status" : "downloaded",
			"stop" : 
			[
				"<|im_end|>"
			],
			"stream" : true,
			"system_template" : "<|im_start|>system\n",
			"temperature" : 0.7,
			"text_model" : false,
			"tfs_z" : 1.0,
			"top_k" : 40,
			"top_p" : 0.9,
			"typ_p" : 1.0,
			"user_template" : "<|im_end|>\n<|im_start|>user\n",
			"version" : "1"
		}
	],
	"object" : "list",
	"result" : "OK"
}

[LazyYuuki]

Hi @MalfuncEddie , I believe that the problem might be because you still need to input some random API key into the API key input for the request to work. This is because OpenAI-compatible API require to always have an API key value.

Do let me know if it works for you!