上圖為用 MegaPortal 加載不同的 Stable Diffusion 模型，然后輸入關于鋼鐵俠的提示語生成的圖片。

The figure above shows images generated by inputting prompts about Iron Man using different Stable Diffusion models loaded with MegaPortal.

什么是 MegaPortal / What is MegaPortal?

MegaPortal 是一個為蘋果設備設計的易用 AI 模型加載工具，下面是我讓 ChatGPT 幫我想的三點賣點：

1. 易于使用：MegaPortal代碼段由基本的、易于使用的可視塊組成，可以通過低成本進行配置。
2. AI 適用于所有人：MegaPortal 不僅面向 AI 專家和開發人員，也面向非技術用戶。它是一個用戶友好型工具，可以被任何希望快速測試、利用或分享 AI 模型的人使用。
3. 本地隱私優先：所有的 AI 模型都在您的設備上本地運行，所有的輸入數據也在本地處理。MegaPortal 不會從您或您的設備中收集任何數據。

詳情請訪問 MegaPortal 的文檔站點：https://docs.getmegaportal.com/ 。同時，MegaPortal 是一個小巧（4.5 MB, gzipped）且免費的應用 ??。

MegaPortal is an easy-to-use AI model loader designed for Apple devices. Here are three key selling points that ChatGPT helped me come up with:

1. Easy to Use: MegaPortal snippets are composed of basic, easy-to-use visual blocks that can be configured with minimal effort.
2. AI Accessible to All: MegaPortal is intended for use by not just AI experts and developers, but also non-technical users. It is a user-friendly tool that can be utilized by anyone looking to quickly test, utilize, or share AI models.
3. Local and Privacy First: All AI Models Run Locally on your device, and all input data is processed locally as well. MegaPortal does not collect any data from you or your device.

For more details: https://docs.getmegaportal.com/

有什么用 / What is it used for?

MegaPortal 是一個文檔類型軟件，執行文件稱為 Snippet，Snippet 由若干個 Block 組成，例如下面是一個 Snippet 的例子：

"MegaPortal is a document-type software, and its executable files are called snippets. Snippets are composed of multiple Blocks, as shown in the following example:

這個 Snippet 實現了一個人臉濾鏡，以圖片作為輸出，先經過 AnimeGANv2 將圖片變成 Anime 風格的圖片，由于這個模型輸出的是 512X512 大小的圖片，再通過 SRGAN 超解析模型變成 2048X2048 大小的圖片。相關模型介紹：

This snippet implements a filter, generating an image output that goes through the AnimeGANv2 model to transform the image into an anime-style image. Since this model outputs images with a size of 512x512, the SRGAN super-resolution model is then applied to upscale the image to a size of 2048x2048. Here are some brief introductions to the relevant models:

1. ??https://github.com/bryandlee/animegan2-pytorch??
2. ??https://github.com/dongheehand/SRGAN-PyTorch??

為了進一步介紹 MegaPortal 的用途，下面介紹三個本人配置的 Snippet：

To further illustrate the usage of MegaPortal, below are three Snippets that I have configured:

1. 第一個視頻演示的是上述人臉濾鏡的效果，配置原理已經介紹過；
2. 由于本人是原神玩家，第二個視頻配置了一個用于推薦原神抽卡的 Snippet。它接受米游社的角色列表圖片作為輸入，通過 MegaPortal 的 Visual Text Recognition Block^[1] 處理后得到圖片中的文字信息。接著，再通過一個 Javascript Execution Block^[2] 進行處理。這個 JS Block 主要承擔相似文字擬合（由于原神生僻字太多，例如“云堇”可能會被識別成“云革”，需要進行擬合）、推薦和表單展示邏輯功能。順帶一提，Javascript Execution Block 是通過 VSCode 插件進行書寫的，插件配備類型提示和編譯功能。
3. 第三個視頻則是配置了一個利用 Stable Diffusion 模型來通過文字生成圖片的 Snippet。第一個 Block 是 Javascript Execution Block^[3]，用來展示表單和將表單輸入內容傳遞給第二個 Block。第二個 Block 則加載一個起步為 2G 的 SD 模型來生成圖片。需要注意的是，由于 SD 模型較大，第一次運行需要一定的加載時間。同時，雖然上面視頻演示的是 iPhone 的運行結果，但實際上，目前我只成功地跑起了一個標準的 SD 模型，其他模型運行時都會崩潰。這也是我在“尋求幫助”部分想要求助的內容。
4. The first video demonstrates the effect of the above-mentioned filter, and how it worked has been introduced earlier.
5. As a Genshin Impact player, the second video introduced a Snippet for recommending Genshin Impact gacha. It accepts the a picture of character list from miHoYo's app as input, processes the text information in the image through MegaPortal's Visual Text Recognition Block^[4], and then processes it through a Javascript Execution Block^[5]. This JS Block mainly handles similar text fitting (due to the many rare characters in Genshin Impact, such as "云堇" may be recognized as "云革", which needs to be fitted), recommendation, and display results. By the way, the Javascript Execution Block is written through a VSCode plug-in, which is equipped with type prompts and compilation functions.
6. The third video demonstrates a Snippet configured to generate images from text using the Stable Diffusion model. The first Block is a Javascript Execution Block^[6], which is used to display the form and pass the input content to the second Block. The second Block loads an SD model sized as at least 2G to generate images. It should be noted that, due to the large size of the SD model, it takes some time to load when it is first run. Also, although the video above demonstrates the results of running on an iPhone, in reality, I have only been able to successfully run a standard SD model, and other models crash during runtime. This is also what I am seeking help with in the "Seeking Help" section.

在應用的 Snippet Center 中可以挖掘更多的有趣的 Snippet，你也可以配置自己的 Snippet，通過分享 Snippet 文件或者發布到 Snippet Center 中供其他用戶下載。

In the Snippet Center of the application, you can discover more interesting Snippets, and you can also configure your own Snippets to share the Snippet file or publish it to the Snippet Center for other users to download.

使用教程 / How to use it

本環節主要以配置一個 Stable Diffusion 的 Snippet 為例，向大家講解 MegaPortal 的使用方法。

In this section, we will mainly use configuring a Stable Diffusion Snippet as an example to introduce the usage of MegaPortal.

系統要求 / System Requirement

為了使用 MegaPortal 的全部功能（特指 Stable Diffusion），請將設備至少升級至 iOS/iPadOS 16.2, macOS 13.1.

To use all the features of MegaPortal, especially Stable Diffusion, please upgrade your device to at least iOS/iPadOS 16.2 and macOS 13.1.

下載 / Download

請訪問 https://www.getmegaportal.com/ 下載最新版本，目前 iOS 和 macOS 版本都可下載.

Please visit https://www.getmegaportal.com/ to download the latest version. Currently, both iOS and macOS versions are available for download.

配置一個 AI 生成圖片 Snippet / Configure an AI image Generation Snippet

下面將在 macOS 下為例，講述配置一個 Snippet 的過程。

The following will describe the process of configuring a Snippet on macOS.

第一步，打開 MegaPortal，點擊新建文檔，創建一個空文件。然后，下載一個 Stable Diffusion 的 Snippet 作為模板配置，然后雙擊打開這個文件：

Step 1: Open MegaPortal and click "New Document" to create a new file. Then, download a Stable Diffusion Snippet as a template for configuration, and double-click to open the file.

由于模型中自帶 Stable Diffusion 模型，打開后程序會自動開始下載相關模型，模型下載需要一定的時間。如果網絡不好，請先關閉程序，在下面「Stable Diffusion 模型下載」部分用下載工具先下載一個模型。

As the model comes with a prepared Stable Diffusion model, the program will automatically start downloading related models when opened, which may take some time. If the network is not good, please close the program and download a model using a download tool in the "Stable Diffusion Model Download" section below.

當你下載完模型后，可以點開 AI Model Application 的 Block 進行配置，配置完后點「Save」保存，由于之前的下載可能在進行中，目前只能完全關閉 MegaPortal 才能中斷下載。

After downloading the model, you can click on the AI Model Application block to configure it. Once configured, click "Save" to save the changes. Since the previous download may still be in progress, you may need to completely close MegaPortal to interrupt the download.

重新打開文件，點「播放」按鈕即可運行這個 Snippet：

To run this Snippet, you can reopen the file and click on the "Play" button.

單擊右鍵就可以將圖片保存至 Photos 應用。

Right-click on the image and you can save it to the Photos app.

清理緩存 / Clear Cache

運行一段時間之后，應用會緩存很多的模型文件，可以通過下面步驟清理緩存：

1. 點擊「More」按鈕；
2. 點擊 Configuration 按鈕；
3. 點擊「Local Caches」按鈕；
4. 點擊「Delete All」或者長按 / 右鍵某一條目進行刪除；

To clear the cache of MegaPortal, you can follow these steps:

1. Click the "More" button;
2. Click the "Configuration" button;
3. Click the "Local Caches" button;
4. Click "Delete All" or long press/right click on a specific item to delete it.

緣起 / Source of Inspiration

回憶起之所以設計這個軟件，我覺得有三個契機：

1. 在過去一年時間我有幸接觸到了 Standford 的 CS193P^[7] 課程，學習了 SwiftUI 的開發，同時由于自己是前端通道的同學，所以同時順便了解到了 iOS 上的 Web 容器 WKWebView 和 JS Runtime JavascriptCore 的細節使用方法。
2. 由于過于一段時間參與了 aPaaS 項目的相關工作，在 aPaaS 項目得到了一些認知，這些認知幫助我能夠更好得去抽象一個針對具體領域的效率提高工具的設計。
3. 然后最重要的可能是，由于我是原神玩家，在原神中會遇到計算角色強度相關的問題，而計算角色強度需要用戶將自己角色面板的數字輸入一些小程序 / 網頁中才能實現，這個過程比較繁瑣，所謂懶惰是第一生產力，所以我就想著可以在手機上寫一個程序，能夠以視頻或者圖片作為輸入，快速計算出角色強度。而解決這個問題第一步需要解決 OCR 的問題，在查閱和在 iOS 實現 OCR 相關功能的過程中，發現蘋果 VNRecognizeTextRequest 乃至整個 AI 應用的相關 API 是一個整體的體系，復用程度比較高，可以稍微整理產品化。

I recall three factors that inspired me to design this software:

1. In the past year, I had the opportunity to study SwiftUI development through Stanford's CS193P course. At the same time, as a front-end engineer, I also learned about the details and usage of WKWebView and JS Runtime JavascriptCore on iOS.
2. Due to my involvement in an aPaaS project for some time, I gained some knowledge that helped me better abstract the design of an efficiency improvement tool for a specific domain.
3. The most important factor may be that I am a player of Genshin Impact, a game in which calculating the strength of characters involves inputting numbers from the character panel into some programs/websites. This process is quite cumbersome, and。I thought of creating a program on the phone that could quickly calculate the strength of characters using videos or images as input. Solving this problem first requires solving the OCR problem. While researching and implementing OCR-related functionality on iOS, I discovered that Apple's VNRecognizeTextRequest and the entire AI application-related APIs form a system with a relatively high degree of reuse, which can be productized with some organization.

游戲中的原圖

The original image in the game.

Text Recognition Block 得到的效果

The result obtained by the Text Recognition Block.

得到上述的結果后還需要經由一個 Javascript Block 來計算角色強度。

BTW，不道德曬抽卡：

20發大保底出雙金，夜蘭你是愛我滴

20發出武器

目前進展和后續功能

目前剛添加完 Stable Diffusion 功能，后續有空的話會為 Stable Diffusion 模型應用添加 image2image 功能，也就是下圖中的功能：

尋求幫助 / Seeking Help

發這個帖的最重要目的是目前 Stable Diffusion 功能在 iPhone 上穩定運行仍未解決，大部分我轉換的模型在 iPhone 上運行都會崩潰，而 MegaPortal 設計之初就是希望能夠讓用戶在 macOS 上開發調試，然后將 Snippet 通過 iCloud 等方式同步到 iPhone 上，在 iPhone 上運行或者分享給別的用戶，所以所有模型能夠在 iPhone 上運行尤其重要，即使 Stable Diffusion 只能運行在性能較好的 iPhone 上。

同時，由于本人精力相對有限，完全吃透 Stable Diffusion 乃至 Pytorch 生態對于我來說難度非常大，所以希望有 iOS 開發經驗和 Pytorch + Stable Diffusion 開發經驗的的同學能夠幫忙一起來看看這個問題 ??。

下面附帶兩組模型文件幫忙定位：

The most important purpose of this post is that the Stable Diffusion feature is still not stable on the iPhone, and most of the models I convert will crash when running on the iPhone. MegaPortal was designed to allow users to develop and debug on macOS, and then sync the Snippet to the iPhone via iCloud or other means to run on the iPhone or share with other users. Therefore, it is particularly important that all models can run on the iPhone, even if Stable Diffusion can only run on higher-performance iPhones.

At the same time, due to my limited energy, it is very difficult for me to fully understand Stable Diffusion and the Pytorch ecosystem, so I hope that fellow developers with experience in iOS development and Pytorch + Stable Diffusion development can help me to identify this issue.

Attached are two sets of model files to help with debugging.

會崩潰的模型 / crashing：

1. ??https://model.getmegaportal.com/classicAnim-v1-einsum_compiled.zip??
2. ??https://huggingface.co/nitrosocke/classic-anim-diffusion/blob/main/classicAnim-v1.ckpt??

不會崩潰的模型 / not crashing：

1. ??https://model.getmegaportal.com/coreml-stable-diffusion-2-1-base_split_einsum_compiled.zip??

Stable Diffusion 模型下載 / Model Download

根據網上的開源 SD 項目，轉換好了對應的 CoreML 格式的模型文件，大家可以根據需求下載。

1. 由于為了節省空間和帶寬，大部分模型中沒有帶有 Safety Checker，請不要產生 NSFW 內容 ??
2. 模型大小為 2G~4G，用公司 VPN 下載的話速度尚可。

The CoreML format models corresponding to the open-source SD project have been converted and are available for download as needed.

1. Due to space and bandwidth constraints, most models do not come with a Safety Checker. Please refrain from creating NSFW content ??.
2. The models range in size from 2GB to 4GB and can be downloaded at reasonable speeds using the company VPN.

Ghibli Style / 吉卜力風格

Links / 鏈接

1. Download / 下載：https://model.getmegaportal.com/ghibli-diffusion-v1-einsum_compiled.zip
2. Source / 源項目：https://huggingface.co/nitrosocke/Ghibli-Diffusion

Anime Style / 動漫風格

本人最喜歡的一類風格

Links / 鏈接

1. Download / 下載：https://model.getmegaportal.com/8528-diffusion2_split-einsum_compiled.zip
2. Source / 源項目：https://huggingface.co/852wa/8528-diffusion

Elden Ring Style / 指環王老頭環風格

Links / 鏈接

1. Download / 下載：https://model.getmegaportal.com/eldenRing-v3-pruned-einsum_compiled.zip
2. Source / 源項目：https://huggingface.co/nitrosocke/elden-ring-diffusion

Classic Disney Style / 經典迪士尼風格

Links / 鏈接

1. Download / 下載：https://model.getmegaportal.com/classicAnim-v1-einsum_compiled.zip
2. Source / 源項目：https://huggingface.co/nitrosocke/classic-anim-diffusion

Redshift Style / RedShift 風格

Links / 鏈接

1. Download / 下載：https://model.getmegaportal.com/redshift-einsum_compiled.zip
2. Source / 源項目：https://huggingface.co/nitrosocke/redshift-diffusion

Spideverse Style / 蜘蛛人：新宇宙風格

Links / 鏈接

1. Download / 下載：https://model.getmegaportal.com/spiderverse-v1-pruned-einsum_compiled.zip
2. Source / 源項目：https://huggingface.co/nitrosocke/spider-verse-diffusion

Archer Style / 間諜亞契風格

Links / 鏈接

1. Download / 下載：https://model.getmegaportal.com/archer-v1-einsum_compiled.zip
2. Source / 源項目：https://huggingface.co/nitrosocke/archer-diffusion

Anime Style / 雙城之戰風格

Links / 鏈接

1. Download / 下載：https://model.getmegaportal.com/arcane-diffusion-v3-einsum_compiled.zip
2. Source / 源項目：https://huggingface.co/nitrosocke/Ghibli-Diffusion

Stable Diffusion 2-1 原始 / Original Stable Diffusion 2-1（iPhone 測試可用）

這個模型也是唯一在我拍的 iPhone 14 Pro Max 上測試可用的模型

Links / 鏈接

1. Download / 下載：https://model.getmegaportal.com/coreml-stable-diffusion-2-1-base_split_einsum_compiled.zip
2. Source / 源項目：

1. ??https://huggingface.co/stabilityai/stable-diffusion-2-1??
2. ??https://huggingface.co/pcuenq/coreml-stable-diffusion-2-1-base??

Midjourney v4

Links / 鏈接

1. Download / 下載：https://model.getmegaportal.com/mdjrny-v4-einsum_compiled.zip
2. Source / 源項目：https://huggingface.co/prompthero/openjourney/tree/main

Nitro Diffusion

這是一個多風格支持的模型，支持 archer style, arcane style or modern disney style

Links / 鏈接

1. Download / 下載：https://model.getmegaportal.com/nitroDiffusion-v1-einsum_compiled.zip
2. Source / 源項目：https://huggingface.co/nitrosocke/Nitro-Diffusion

致謝 / Thanks!

1. 謝謝 ChatGPT 幫助翻譯和編程問題的回答；
2. 謝謝 Copilot 結對編程（盡管是雇傭關系 ??）；
3. 感謝社區各種開源的 AI 模型項目，此處不一一致謝了 ??。
4. "Thank you, ChatGPT, for helping me with the translation and programming questions."
5. "Thank you, Copilot, for pair programming with me (even though it's a hired relationship ??)."
6. "I am grateful to various open-source AI model projects in the community. I cannot thank each of them enough ??."

參考資料?

[1]Visual Text Recognition Block: ??https://docs.getmegaportal.com/docs/blocks/visual-text-recognition??

[2]Javascript Execution Block: ??https://docs.getmegaportal.com/docs/blocks/javascript-execution??

[3]Javascript Execution Block: ??https://docs.getmegaportal.com/docs/blocks/javascript-execution??

[4]Visual Text Recognition Block: ??https://docs.getmegaportal.com/docs/blocks/visual-text-recognition??

[5]Javascript Execution Block: ??https://docs.getmegaportal.com/docs/blocks/javascript-execution??

[6]Javascript Execution Block: ??https://docs.getmegaportal.com/docs/blocks/javascript-execution??

[7]CS193P: ??https://cs193p.sites.stanford.edu/??