???? ???, ??? ???, ??? ???? ??? ? ?? GPT-2? ?? ??? ?? ??? ??? ??? ????? ???? ? ???? ?? ???? ????? ??? ???.

??? ?? ???? ?????. ?, ?? ??? ???? ??? ?? ???? ???? ???? ?? ??? ?????.

0??: ?? ??

    !pip install torch transformers accelerate bitsandbytes psutil

    from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
    import torch
    import time
    import gc

    def get_memory_usage():
        return torch.cuda.memory_allocated() / 1e6 if torch.cuda.is_available() else 0


    device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
    model_name = "gpt2"
    input_text = "Once upon a time"

---

1??: ?? – ?? ???(FP32)

??? ?? ??? 32?? ?? ??? ???(FP32)? GPT-2? ?????. ?? ??? "?? ??" ??? ?? ????? ??? ??????.

• ???: FP32 ??? ???? 511MB? GPU ???? ?????.
• ??: ???? “Once upon a time”?? 50?? ??? ???? ? 1.76?? ????.
• ?? ? ??: ??? ??? ??? 458MB? ???? ?? ????.

FP32? ????? ??? ???.

    # Load tokenizer and base model
    tokenizer = AutoTokenizer.from_pretrained(model_name)
    print(f"Pre-load memory: {get_memory_usage()} MB")

    # Full precision model
    model_fp32 = AutoModelForCausalLM.from_pretrained(model_name).to(device)
    print(f"Post-load memory: {get_memory_usage()} MB")  # 511.15 MB

    # Inference measurement
    inputs = tokenizer(input_text, return_tensors="pt").to(device)
    start_time = time.time()
    output = model_fp32.generate(**inputs, max_length=50)
    inference_time = time.time() - start_time  # 1.76s

    # Cleanup protocol
    del model_fp32, inputs
    gc.collect()
    torch.cuda.empty_cache()

---

2??: ?? ?? – 8?? ???(INT8)

???? ???? ?? ??? ?? ??? ???? 8?? ???? ?????. ??? ??????:

• ???: INT8 ??? FP32?? 187MB? ???? 63% ? ????.
• ??: ??? 1.38?? ????? 22% ?????.
• ?? ? ??: ?? ? ???? 139MB? ?????.

? ??? ? ??? ??? ??? ??????. ??? ?????.

    # 8-bit configuration
    quant_config_8bit = BitsAndBytesConfig(load_in_8bit=True)

    print(f"Pre-load memory: {get_memory_usage()} MB")  # 9.18 MB
    model_int8 = AutoModelForCausalLM.from_pretrained(
        model_name, 
        quantization_config=quant_config_8bit
    )

    # Dynamic input handling
    inputs_int8 = tokenizer(input_text, return_tensors="pt").to(model_int8.device)
    start_time = time.time()
    output = model_int8.generate(**inputs_int8, max_length=50)  # 1.38s

---

3??: ???? ?? – 4?? ???(INT4)

?? ? ???????. 4?? ???? ???? ???? ?? ?? ???? ???? ????? ???? ?? 16?? ?? ???? ?????.

• ???: INT4 ??? ??? 149MB? FP32?? 71% ? ?????.
• ??: ?? ??? 1.08?? ????, ?? FP32? ?? 39% ??
• ?? ? ??: ???? 58MB? ?????. ?? ??? ?????.

??? ??? ???? ????. ??????.

    # 8-bit configuration
    quant_config_8bit = BitsAndBytesConfig(load_in_8bit=True)

    print(f"Pre-load memory: {get_memory_usage()} MB")  # 9.18 MB
    model_int8 = AutoModelForCausalLM.from_pretrained(
        model_name, 
        quantization_config=quant_config_8bit
    )

    # Dynamic input handling
    inputs_int8 = tokenizer(input_text, return_tensors="pt").to(model_int8.device)
    start_time = time.time()
    output = model_int8.generate(**inputs_int8, max_length=50)  # 1.38s

---

???: ???? ???

???? ??? ????. ???? ??? ?? ???? ???? ??? ? ??? ???? ??? ??? ?? ?? ????? ??? ?? ?? ????. ??? ?? ?? ???? ?? ???.

• ??? ???:FP32: 511MB → INT8: 187MB → INT4: 149MB.

??: ??? ? ??? ??? ??? ???? ??? GPU ?? ?? ??? ??? ? ????.

• ?? ??:FP32: 1.76? → INT8: 1.38? → INT4: 1.08?.

??: ???? ???? ??? ???? ??? ??????? ?? ?? ??? ??????.

---

?? ??: ?? ????

????? ???? ???? ?(?: 32?? ?? ???)? ???? ?? ??(8?? ?? 4?? ??)?? ?????. ?:

• FP32? ??? 32??? ???? ??? ?? ??? ????? ??? ???? ?????.
• INT8/INT4 ? ?? ?? ??? ???? ??? ?????? ?? ??????.

bitsandbytes ?????? ???? ???? ?? ???? ?? ???? ??? ???? ?? ???? ?????.

---

??? ??

??? ???? ??? ??????.

• ??? ???(??? ??): FP32? INT8 ? INT4? ???? ??? ???? ??? ??? ?? ?????.
• ?? ??(? ??): FP32?? INT4?? ?? ???? ?? ??? ?????.

?????? ???? ??? ??? ??? ??? AI ???? ?? ???? ?????.

    !pip install torch transformers accelerate bitsandbytes psutil

    from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
    import torch
    import time
    import gc

    def get_memory_usage():
        return torch.cuda.memory_allocated() / 1e6 if torch.cuda.is_available() else 0


    device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
    model_name = "gpt2"
    input_text = "Once upon a time"

---

??? ?

??? ???? ?? GPT-2? ???? ?? ?? ???? ???? ???? ??? ??????. ?? ??? ??? ??? ???? ??? ???? ?? ?? ? ??? ?? ??????.

? ??? ???? ??? ??? ?? ???? ?? ?????. ? 10~15?? ??? ???? ???? ???? ??? ??????.

• ??? ??? 71% ??
• 39% ? ?? ?? ??

???? ??? ?? ?? ???? ????? ??? Google Colab? ?????.

? ??? ???? ? : gptunleashing ??? ?? ???? ?? ?????. ??? ??? PHP ??? ????? ?? ?? ??? ?????!