Title : After updated #348, encountered Jinja2Cpp error
Discussion category :
openvinotoolkit,openvino_build_deploy
Additional information:
After updated #348, encountered Jinja2Cpp error. I tried to add "llm.set_chat_template("<|user|>\n{prompt}</s>\n<|assistant|>\n")" into app.py but didn't solve a problem.
- Error -
(venv) C:\Users\ias\openvino_build_deploy\ai_ref_kits\agentic_llm_rag>python app.py
C:\Users\ias\openvino_build_deploy\ai_ref_kits\agentic_llm_rag\venv\Lib\site-packages\torch\onnx\_internal\registration.py:162: OnnxExporterWarning: Symbolic function 'aten::scaled_dot_product_attention' already registered for opset 14. Replacing the existing function with new function. This is unexpected. Please report it on https://github.com/pytorch/pytorch/issues.
warnings.warn(
INFO:__main__:loading in <llama_index.core.indices.vector_store.base.VectorStoreIndex object at 0x0000021C645C2090>
C:\Users\ias\openvino_build_deploy\ai_ref_kits\agentic_llm_rag\venv\Lib\site-packages\llama_index\core\agent\react\base.py:154: DeprecationWarning: Call to deprecated class ReActAgent. (ReActAgent has been rewritten and replaced by llama_index.core.agent.workflow.ReActAgent.
This implementation will be removed in a v0.13.0 and the new implementation will be promoted to the `from llama_index.core.agent import ReActAgent` path.
See the docs for more information: https://docs.llamaindex.ai/en/stable/understanding/agent/)
return cls(
C:\Users\ias\openvino_build_deploy\ai_ref_kits\agentic_llm_rag\venv\Lib\site-packages\deprecated\classic.py:184: DeprecationWarning: Call to deprecated class AgentRunner. (AgentRunner has been deprecated and is not maintained.
This implementation will be removed in a v0.13.0.
See the docs for more information on updated agent usage: https://docs.llamaindex.ai/en/stable/understanding/agent/)
return old_new1(cls, *args, **kwargs)
Demo is ready!
* Running on local URL: http://127.0.0.1:7860
INFO:httpx:HTTP Request: GET http://127.0.0.1:7860/gradio_api/startup-events "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: HEAD http://127.0.0.1:7860/ "HTTP/1.1 200 OK"
* To create a public link, set `share=True` in `launch()`.
INFO:httpx:HTTP Request: GET https://api.gradio.app/pkg-version "HTTP/1.1 200 OK"
INFO:__main__:log_history ### 🤔 Agent's Thought Process
Traceback (most recent call last):
File "C:\Users\ias\openvino_build_deploy\ai_ref_kits\agentic_llm_rag\venv\Lib\site-packages\gradio\queueing.py", line 626, in process_events
response = await route_utils.call_process_api(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\ias\openvino_build_deploy\ai_ref_kits\agentic_llm_rag\venv\Lib\site-packages\gradio\route_utils.py", line 322, in call_process_api
output = await app.get_blocks().process_api(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\ias\openvino_build_deploy\ai_ref_kits\agentic_llm_rag\venv\Lib\site-packages\gradio\blocks.py", line 2220, in process_api
result = await self.call_function(
^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\ias\openvino_build_deploy\ai_ref_kits\agentic_llm_rag\venv\Lib\site-packages\gradio\blocks.py", line 1743, in call_function
prediction = await utils.async_iteration(iterator)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\ias\openvino_build_deploy\ai_ref_kits\agentic_llm_rag\venv\Lib\site-packages\gradio\utils.py", line 785, in async_iteration
return await anext(iterator)
^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\ias\openvino_build_deploy\ai_ref_kits\agentic_llm_rag\venv\Lib\site-packages\gradio\utils.py", line 776, in __anext__
return await anyio.to_thread.run_sync(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\ias\openvino_build_deploy\ai_ref_kits\agentic_llm_rag\venv\Lib\site-packages\anyio\to_thread.py", line 56, in run_sync
return await get_async_backend().run_sync_in_worker_thread(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\ias\anaconda3\Lib\asyncio\futures.py", line 287, in __await__
yield self # This tells Task to wait for completion.
^^^^^^^^^^
File "C:\Users\ias\anaconda3\Lib\asyncio\tasks.py", line 349, in __wakeup
future.result()
File "C:\Users\ias\anaconda3\Lib\asyncio\futures.py", line 203, in result
raise self._exception.with_traceback(self._exception_tb)
File "C:\Users\ias\openvino_build_deploy\ai_ref_kits\agentic_llm_rag\venv\Lib\site-packages\anyio\_backends\_asyncio.py", line 967, in run
result = context.run(func, *args)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\ias\openvino_build_deploy\ai_ref_kits\agentic_llm_rag\venv\Lib\site-packages\gradio\utils.py", line 759, in run_sync_iterator_async
return next(iterator)
^^^^^^^^^^^^^^
File "C:\Users\ias\openvino_build_deploy\ai_ref_kits\agentic_llm_rag\venv\Lib\site-packages\gradio\utils.py", line 923, in gen_wrapper
response = next(iterator)
^^^^^^^^^^^^^^
File "C:\Users\ias\openvino_build_deploy\ai_ref_kits\agentic_llm_rag\app.py", line 305, in _generate_response
response = agent.stream_chat(chat_history[-1][0])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\ias\openvino_build_deploy\ai_ref_kits\agentic_llm_rag\venv\Lib\site-packages\llama_index_instrumentation\dispatcher.py", line 324, in wrapper
result = func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\ias\openvino_build_deploy\ai_ref_kits\agentic_llm_rag\venv\Lib\site-packages\llama_index\core\callbacks\utils.py", line 42, in wrapper
return func(self, *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\ias\openvino_build_deploy\ai_ref_kits\agentic_llm_rag\venv\Lib\site-packages\llama_index\core\agent\runner\base.py", line 758, in stream_chat
chat_response = self._chat(
^^^^^^^^^^^
File "C:\Users\ias\openvino_build_deploy\ai_ref_kits\agentic_llm_rag\venv\Lib\site-packages\llama_index_instrumentation\dispatcher.py", line 324, in wrapper
result = func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\ias\openvino_build_deploy\ai_ref_kits\agentic_llm_rag\venv\Lib\site-packages\llama_index\core\agent\runner\base.py", line 640, in _chat
cur_step_output = self._run_step(
^^^^^^^^^^^^^^^
File "C:\Users\ias\openvino_build_deploy\ai_ref_kits\agentic_llm_rag\venv\Lib\site-packages\llama_index_instrumentation\dispatcher.py", line 324, in wrapper
result = func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\ias\openvino_build_deploy\ai_ref_kits\agentic_llm_rag\venv\Lib\site-packages\llama_index\core\agent\runner\base.py", line 438, in _run_step
cur_step_output = self.agent_worker.stream_step(step, task, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\ias\openvino_build_deploy\ai_ref_kits\agentic_llm_rag\venv\Lib\site-packages\llama_index_instrumentation\dispatcher.py", line 324, in wrapper
result = func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\ias\openvino_build_deploy\ai_ref_kits\agentic_llm_rag\venv\Lib\site-packages\llama_index\core\callbacks\utils.py", line 42, in wrapper
return func(self, *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\ias\openvino_build_deploy\ai_ref_kits\agentic_llm_rag\venv\Lib\site-packages\llama_index\core\agent\react\step.py", line 841, in stream_step
return self._run_step_stream(step, task)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\ias\openvino_build_deploy\ai_ref_kits\agentic_llm_rag\venv\Lib\site-packages\llama_index\core\agent\react\step.py", line 658, in _run_step_stream
chat_stream = self._llm.stream_chat(input_chat)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\ias\openvino_build_deploy\ai_ref_kits\agentic_llm_rag\venv\Lib\site-packages\llama_index_instrumentation\dispatcher.py", line 324, in wrapper
result = func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\ias\openvino_build_deploy\ai_ref_kits\agentic_llm_rag\venv\Lib\site-packages\llama_index\core\llms\callbacks.py", line 175, in wrapped_llm_chat
f_return_val = f(_self, messages, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\ias\openvino_build_deploy\ai_ref_kits\agentic_llm_rag\venv\Lib\site-packages\llama_index\llms\openvino_genai\base.py", line 396, in stream_chat
prompt = self.messages_to_prompt(messages)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\ias\openvino_build_deploy\ai_ref_kits\agentic_llm_rag\venv\Lib\site-packages\llama_index\llms\openvino_genai\base.py", line 296, in _tokenizer_messages_to_prompt
self._tokenizer.apply_chat_template(
RuntimeError: Exception from C:\Jenkins\workspace\private-ci\ie\build-windows-vs2022\b\repos\openvino.genai\src\cpp\src\tokenizer\tokenizer.cpp:703:
Jinja2Cpp failed to apply chat template. Possible solutions are
* Provide a simplified chat template with set_chat_template().
* Set apply_chat_template to false in GenerationConfig. It's possible to apply the template manually to your prompt before calling generate. For example: <|user|>\n{prompt}</s>\n<|assistant|>\n
Jinja2Cpp's error: bad_expected_access
