FAQ: AI (tyDiffusion)


Installation

•  Do I need a tyFlow PRO license to use tyDiffusion with GPU acceleration?

No, a license is not required to use tyDiffusion’s GPU accelerated functionality. tyDiffusion interfaces with ComfyUI to generate images/animations, and doesn’t place any constraints on ComfyUI’s abilities if a tyFlow PRO license is not found. In general, a tyFlow PRO license is only required to unlock tyFlow’s internal CPU/GPU acceleration - and since ComfyUI is an external application accessed only through a common API, no extrinsic restrictions are placed on its performance.

•  Do I need a GPU to use tyDiffusion?

tyDiffusion works best on a NVIDIA GPU with at least 8GB of VRAM (12GB+ of VRAM is recommended for image generation, and 24GB+ of VRAM is recommended for animation generation). Because AI models tend to be quite large, a lot of VRAM is required to load them onto a GPU. SD 1.5 checkpoints are usually around 2GB in size, and SD XL checkpoints are usually around 6GB in size. Combine the size of those requisite checkpoints with the size of any ControlNet models being used, IP-Adapters, AnimateDiff, etc, and VRAM usage requirements can quickly skyrocket.

If your GPU has less than 8-12GB of VRAM, you can try enabling the low VRAM command line arguments for ComfyUI (tyDiffusion UI > ‘Setup’ tab > ‘Engine’ tab > ‘Advanced’ tab > ‘Additional command line args’), or you can even enable CPU-only mode in the same location (although tyDiffusion will run unbearably slow without proper GPU acceleration).

If you are using an AMD GPU, you can enable DirectML in the advanced engine settings (tyDiffusion UI > ‘Setup’ tab > ‘Engine’ tab > ‘Advanced’ tab), although DirectML will run slower on an AMD GPU than PyTorch will run on a NVIDIA GPU.

•  How can I ensure the version of Python included with tyDiffusion does not interfere with an existing version of Python on my machine?

The embedded version of Python 3.10 installed by tyDiffusion runs in an isolated environment and uses local system variable assignments. The installation of its modules and system variables should not affect existing installations of Python on a user’s machine.

•  I already have ComfyUI installed on my system. Do I need to install it again for tyDiffusion?

While tyDiffusion interfaces with ComfyUI using its standard web API, the version of ComfyUI installed by tyDiffusion includes a variety of custom nodes that have been forked from their original repositories, and tweaked in various ways. For this reason it is not possible to generate images with an alternative ComfyUI installation using tyDiffusion, even though both installations may rely on the same web API syntax.

However, tyDiffusion does include the ability to specify alternative model paths - so you can point it to shared model folders used by other ComfyUI installations, avoiding the need to store redundant/duplicate checkpoints/models/etc on a machine.

•  Do I need to reinstall tyDiffusion each time I update tyFlow?

No. Unless you are specifically instructed to reinstall tyDiffusion after a tyFlow update (either in the tyDiffusion UI, or in the tyFlow version history release notes), you don’t need to re-install tyDiffusion after updating tyFlow to a newer build.

•  How can I re-install tyDiffusion?

Simply running the one-click installer again is enough to completely re-install tyDiffusion, ComfyUI and all of the various other tools included with it. You don’t need to uninstall tyDiffusion prior to running the one-click installer again.

•  How do I uninstall tyDiffusion?

Everything tyDiffusion installs goes into its root folder and external model paths. The default root folder is: C:\ProgramData\tyFlow\tyDiffusion (and the default external model paths are inside the root folder). Deleting the root folder (and external model paths, if you changed them from their defaults) is enough to completely uninstall and remove tyDiffusion from your system.

•  My company does not allow us to install or use software with AI-based features. How can I remove these features from tyFlow?

tyFlow (ie, the actual .dlo file you install in order to use and run tyFlow) does not itself contain any AI models, algorithms, generators, etc. Those features are only unlocked when you manually complete the tyDiffusion installation from the tyDiffusion UI. That installation process downloads and installs a Python-based AI pipeline on your machine, which tyDiffusion then interfaces with using a common API in order to generate images/animations. So, without manually completing the tyDiffusion installation process, no actual AI tools will be present on the machine or available to the end user.

Furthermore, you can completely restrict a machine from accessing any AI features within tyFlow by defining the following system environment variable: TYFLOW_NO_AI=1. If a user attempts to install or use tyDiffusion when that environment variable is present, a popup notification will alert them to the fact that all of tyFlow’s AI features are disabled, due to the presence of that environment variable on the system.

•  Is it safe to run ComfyUI on my machine? If tyDiffusion downloads and executes 3rd party code in order to perform its operations, how do I know it does not contain malware?

There is always a risk that comes with executing 3rd party code downloaded from the internet onto a local machine. During its installation process, tyDiffusion draws from many different sources to acquire the huge number of Python modules required to run ComfyUI, and it is practically impossible to vet every single line of code in every single module, in order to ensure that no malware has been injected into it by bad actors.

However, several important steps have been taken in order to minimize risks to users of tyDiffusion:

  • During the installation process, ComfyUI and all of the custom nodes that tyDiffusion uses are downloaded from open-source repositories that have been forked from their official sources. These repositories can all be found on the tyDiffusion GitHub page. Because these repositories have been forked and are only synced manually, they are immune to zero-day exploits that might be added in the future to their original counterparts. In other words, if malicious code is added to the official ComfyUI repository (found here), or any other official custom node repository, users who install tyDiffusion after that point will not be exposed to those exploits because the tyDiffusion installer does not pull from those official repositories, but instead from forked repositories that are not automatically synced to their official counterparts.

  • All requirements.txt files (used by Python’s “pip” module to install external Python packages) in the various tyDiffusion repositories only pull packages from official, trusted Python repositories, avoiding the exploit discovered here.

  • AI models that can be downloaded from the tyDiffusion model library have all been hand-picked for quality, and are all stored in controlled tyDiffusion HuggingFace datasets. The safetensor format is also the preferred format for all models in the tyDiffusion model library, with only a few trusted exceptions.

Once again, it is not possible to completely eliminate the threat of malicious code across the hundreds of 3rd party Python modules required to run ComfyUI, but users should feel somewhat at-ease knowing that these steps have been taken in order to minimize many of the potential risks involved. Furthermore, users should always run their own virus scan or threat protection software locally in order to further alleviate risks involved with executing any/all software.

•  I get an error during installation, or the console window closes before installation is finished, or the console window opens and then closes immediately during installation or image generation, etc. How can I debug these errors?

Due to that fact that tyDiffusion runs ComfyUI as a separate process, it can sometimes be tough to track down problems that occur on some machines during installation or image generation, especially because errors usually cause the console window to close immediately (so any error messages that may get printed to the console log disappear from view). In tyFlow v1.112, an option was added in the advanced engine settings (tyDiffusion UI > ‘Setup’ tab > ‘Engine’ tab > ‘Advanced’ tab) to enable “install/engine file logging”. By enabling this setting and then running the installer or engine again, everything printed to the console window will also be printed to corresponding log files on disk - if an error occurs the console window may still close immediately, but its contents will be saved to a log file (which can then be sent to support@tyflow.com for further debugging assistance).

If you are unable to access the tyDiffusion UI for some reason (ex: the ComfyUI console windows closes immediately after opening the tyDiffusion UI and 3ds Max becomes unresponsive, preventing you from being able to enable file logging mode), you can also take these steps to further debug the cause of the problem:

  • Navigate to the tyDiffusion root folder in Windows explorer (the default root folder is: C:\ProgramData\tyFlow\tyDiffusion)

  • Hold SHIFT and right-click in the folder, then select “Open PowerShell window here”

  • Type “cmd” (no quotes) and hit ENTER.

  • Type “run_ComfyUI.bat” (no quotes) and hit ENTER.

  • ComfyUI will attempt to intialize and (presumably) fail again - but instead of the console window closing immediately like it does within 3ds Max, the error which is causing the problem should remain visible. If the error is not something you understand how to fix on your own, please send a screenshot of the error message to support@tyflow.com.

If 3ds Max becomes unresponsive while performing a tyDiffusion task, holding SHIFT+ESC on the keyboard can usually return control to the user.

Either way, there are a few issues which can cause the installer or engine to fail during launch:

  • Folder permissions preventing 3ds Max from modifying files in the tyDiffusion target directory may interfere with the installer/engine. The simplest fix is to simply run 3ds Max as Administrator. If that’s not a possibility, you can either set a different folder as the tyDiffusion root directory (from within tyDiffusion’s UI), or modify the target folder’s permissions to allow 3ds Max (running in non-Administrator mode) to read/write/modify the folder’s contents without restrictions.

  • If you click your mouse within the console window at any point during its execution, execution of the batch commands will be paused (which will stall the installer/engine). A sign that you may have accidentally clicked within the window is the appearance of a large block character in the window where you clicked the mouse. You can fix this problem immediately by clicking anywhere in the window again (to ensure the window is in focus) and pressing “Enter” on your keyboard to resume execution.

  • If you prematurely close the console window before it’s finished executing its commands, the installation process will fail. Make sure you don’t close the console window while it’s running.

•  When I try to download models from tyDiffusion’s model library, the download manager shows an error saying the URL is invalid or cannot be reached. What do I do?

tyDiffusion uses very standard cURL commands to download models through its model library and download manager, so if a download fails there are really only two potential causes:

  • The URL itself if offline. This would entail that either HuggingFace is offline, or the specified model has been removed from HuggingFace. You can validate whether or not this is the case by navigating to the tyDiffusion HuggingFace repository and checking to see if the desired file is present in the corresponding dataset.

  • Something on your machine is preventing 3ds Max from connecting to the internet. This could be a firewall blocking 3ds Max from connecting to the internet, or a VPN/proxy used by your machine that is not configured to allow connections from 3ds Max to go through it.

The first case is much less likely than the second. In the second case, if you are unable to configure your machine to allow an internet connection to go through 3ds Max, you can manually download all desired files from the HuggingFace repo linked above and place them in the corresponding external model paths defined within tyDiffusion’s UI.

•  Why are models that I’ve downloaded not appearing in the tyDiffusion UI and/or shown as missing?

There are a few reasons why tyDiffusion may not be able to find models you’ve downloaded:

  • Models needs to be placed in the appropriate model path. Model paths can be defined in the ‘Setup’ tab > ‘Paths’ > tab. If you download models using tyDiffusion’s download manager, models will be placed in the proper paths automatically.

  • Models need appropriate read permissions. If a model file or folder has read permissions disabled for the current Windows user, tyDiffusion/ComfyUI will not be able to see or load it. Running 3ds Max as Administrator may fix this issue if read permissions are only disabled for non-Admin users, but it is possible to disable read permissions for Admin users as well - so if you find that tyDiffusion can’t see your model even if you’re running 3ds Max as Administrator, that could explain why.

  • tyDiffusion tells ComfyUI where models are by modifying the following yaml file: [tyDiffusion root]\Engines\ComfyUI\ComfyUI\extra_model_paths.yaml (ComfyUI loads this file during startup). If tyDiffusion is unable to create or modify that file (due to it missing appropriate read/write permissions, for example), then ComfyUI may not be able to find the models that tyDiffusion tells it to use. Check the read/write permissions of that file and check the contents of that file using a text editor to ensure it contains your custom tyDiffusion model paths.

tyDiffusion modifies that yaml file each time ComfyUI is started, so users should not modify if themselves, otherwise their changes will be overwritten.

When ComfyUI is started by tyDiffusion, its console window will display messages during startup to confirm various user settings. Custom paths loaded by ComfyUI will be printed with the following format: “Adding extra search path [path type] [path]“. Double-check that ComfyUI is displaying your custom paths in that manner, during startup, to ensure it’s trying to load your models from the correct place. It’s also a good idea to check that no other errors are displayed during startup, to ensure nothing else is preventing ComfyUI from loading properly.

Generation

•  Why isn’t the Depth ControlNet affecting images/animations I generate?

During initial tests of tyDiffusion it was discovered that some machines have issues relaying viewport depth information through 3ds Max’s Nitrous API. The cause of the issue is unknown (potentially related to certain GPU hardware and/or GPU drivers), but the effect of the issue is that some users may find that tyDiffusion’s Depth ControlNet does not function properly when the ControlNet source is set to “viewport depth”. The current workaround/solution to this problem is to set the Depth ControlNet source to something other than “viewport depth” (ex: set it to “viewport color” instead) and enable one of the depth preprocessors in the ControlNet’s preprocessor menu (ex: “depth-anything”). This will generate depth information from the viewport’s color buffer using a trained AI model, which is not as accurate as using the depth buffer directly, but can still produce acceptable results.

The cause of this issue was recently discovered - if you have anti-aliasing enabled in the viewport tyDiffusion will not be able to extract the depth buffer from the view. This issue is solved in tyFlow v1.113+. If you are using an older build of tyFlow, disable viewport anti-aliasing to restore depth buffer access.

•  When I try to use the IP-Adapter ControlNet, the ComfyUI console window displays an error about a missing clip_vision model. How do I fix this?

tyDiffusion’s IP-Adapter module expects “clip_g.pth” to be present in the following directory: [External ControlNet models]\preprocessors\clip_vision (where [External ControlNet models] is the path defined in ‘Setup tab’ > ‘Paths tab’). The tyDiffusion download manager will automatically place “clip_g.pth” in that directory if you download the IP-Adapter ControlNet models from the model library - however, if you manually downloaded your ControlNet models or moved your models/folders around, it may not be able to find that file. By restoring “clip_g.pth” to that directory, the ComfyUI clip_vision error will no longer occur. If “clip_g.pth” is not present on your machine, it can be download from the tyDiffusion HuggingFace repo.

•  Why aren’t the Pose ControlNets affecting images/animations I generate?

The Pose ControlNets (OpenPose/DensePose) are fairly sensitive to the type of inputs you provide them. The corresponding preprocessor models were trained on photographs of humans (or animals, if you’re using the animal model), so if you provide inputs which do not resemble photographic representations of humanoids/animals, they may not detect any poses therein. If you are posing characters rigged with Character Studio Bipeds, you can change the Source mode to “Biped to OpenPose” for a perfect conversion from Biped skeleton to OpenPose skeleton - this results in more accurate pose-dependent image/animation generation.

•  Why aren’t the motion LoRAs I’m using having an effect on my AnimateDiff animations?

AnimateDiff motion LoRAs are only supported by v2 of the AnimateDiff motion module. Make sure you’ve selected v2 of the module if you’re using motion LoRAs.

•  Why do my generated animations look worse than my generated images, even though all of my other settings are the same?

Animations generated with AnimateDiff may differ greatly in quality from images generated with the same overall settings, due to the way in which AnimateDiff motion modules affect output. AnimateDiff motion modules were trained on much lower quality images than base SD models, and use various methods to improve temporal consistency that may have a big impact on overall output image quality. ControlNets may also behave somewhat differently when AnimateDiff is enabled. The best way to improve the quality of animations generated with AnimateDiff is to enable an upscaler to boost their detail and resolution (‘Hires fix’ works well in this regard, but may require a lot of time and VRAM to run).